[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:05:09 dist-test-slave-7mqr krb5kdc[5128](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:05:09 dist-test-slave-7mqr krb5kdc[5128](info): set up 2 sockets
Sep 25 22:05:09 dist-test-slave-7mqr krb5kdc[5128](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260925 22:05:10.072459 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:05:10 dist-test-slave-7mqr krb5kdc[5128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:05:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:05:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.0.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
I20260925 22:05:10.190518 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:34305
--webserver_interface=127.5.0.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:32843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:34305
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:10.273558 5141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:10.273813 5141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:10.273854 5141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:10.275018 5141 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:05:10.275084 5141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:10.275116 5141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:10.275151 5141 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:05:10.275173 5141 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:05:10.276731 5141 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:32843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:34305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:34305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:10.277037 5141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:10.277302 5141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 22:05:10.282858 5141 server_base.cc:1034] running on GCE node
W20260925 22:05:10.282831 5149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:10.282831 5146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:10.282819 5147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:10.283334 5141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:10.283663 5141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:10.284823 5141 hybrid_clock.cc:648] HybridClock initialized: now 1790373910284806 us; error 51 us; skew 500 ppm
Sep 25 22:05:10 dist-test-slave-7mqr krb5kdc[5128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373910, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:10.287343 5141 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:05:10.287807 5141 webserver.cc:460] Webserver started at http://127.5.0.126:38635/ using document root <none> and password file <none>
I20260925 22:05:10.288129 5141 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:10.288208 5141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:10.288386 5141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:10.289467 5141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "7df3b53b87b3418dbc85b72d48247292"
format_stamp: "Formatted at 2026-09-25 22:05:10 on dist-test-slave-7mqr"
server_key: "143dafaed192f06e2c7bc35b7f47479f"
server_key_iv: "f869be051df28365f778c0a9c90ff025"
server_key_version: "encryptionkey@0"
I20260925 22:05:10.289851 5141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "7df3b53b87b3418dbc85b72d48247292"
format_stamp: "Formatted at 2026-09-25 22:05:10 on dist-test-slave-7mqr"
server_key: "143dafaed192f06e2c7bc35b7f47479f"
server_key_iv: "f869be051df28365f778c0a9c90ff025"
server_key_version: "encryptionkey@0"
I20260925 22:05:10.291225 5141 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:05:10.292039 5156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:10.292215 5141 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:05:10.292313 5141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "7df3b53b87b3418dbc85b72d48247292"
format_stamp: "Formatted at 2026-09-25 22:05:10 on dist-test-slave-7mqr"
server_key: "143dafaed192f06e2c7bc35b7f47479f"
server_key_iv: "f869be051df28365f778c0a9c90ff025"
server_key_version: "encryptionkey@0"
I20260925 22:05:10.292393 5141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:10.301776 5141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:10.303617 5141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:10.303792 5141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:10.307711 5141 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:34305
I20260925 22:05:10.307746 5208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:34305 every 8 connection(s)
I20260925 22:05:10.308117 5141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:05:10.308957 5209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:10.311170 5209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7df3b53b87b3418dbc85b72d48247292: Bootstrap starting.
I20260925 22:05:10.311755 5209 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7df3b53b87b3418dbc85b72d48247292: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:10.312091 5209 log.cc:824] T 00000000000000000000000000000000 P 7df3b53b87b3418dbc85b72d48247292: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:10.312740 5209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7df3b53b87b3418dbc85b72d48247292: No bootstrap required, opened a new log
I20260925 22:05:10.314265 5209 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7df3b53b87b3418dbc85b72d48247292 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7df3b53b87b3418dbc85b72d48247292" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34305 } }
I20260925 22:05:10.314394 5209 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7df3b53b87b3418dbc85b72d48247292 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:10.314415 5209 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7df3b53b87b3418dbc85b72d48247292 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7df3b53b87b3418dbc85b72d48247292, State: Initialized, Role: FOLLOWER
I20260925 22:05:10.314581 5209 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7df3b53b87b3418dbc85b72d48247292 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7df3b53b87b3418dbc85b72d48247292" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34305 } }
I20260925 22:05:10.314642 5209 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7df3b53b87b3418dbc85b72d48247292 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:05:10.314699 5209 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7df3b53b87b3418dbc85b72d48247292 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:05:10.314749 5209 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7df3b53b87b3418dbc85b72d48247292 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:10.315289 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 5141
I20260925 22:05:10.315308 5209 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7df3b53b87b3418dbc85b72d48247292 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7df3b53b87b3418dbc85b72d48247292" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34305 } }
I20260925 22:05:10.315399 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:05:10.315418 5209 leader_election.cc:302] T 00000000000000000000000000000000 P 7df3b53b87b3418dbc85b72d48247292 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7df3b53b87b3418dbc85b72d48247292; no voters:
I20260925 22:05:10.315601 5209 leader_election.cc:288] T 00000000000000000000000000000000 P 7df3b53b87b3418dbc85b72d48247292 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:05:10.315625 5212 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7df3b53b87b3418dbc85b72d48247292 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:10.315743 5212 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7df3b53b87b3418dbc85b72d48247292 [term 1 LEADER]: Becoming Leader. State: Replica: 7df3b53b87b3418dbc85b72d48247292, State: Running, Role: LEADER
I20260925 22:05:10.315874 5209 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7df3b53b87b3418dbc85b72d48247292 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:05:10.315915 5212 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7df3b53b87b3418dbc85b72d48247292 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7df3b53b87b3418dbc85b72d48247292" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34305 } }
I20260925 22:05:10.316221 5121 external_mini_cluster.cc:1428] Setting key 3e178584fbb8da440651e971556d6db5
I20260925 22:05:10.316355 5214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7df3b53b87b3418dbc85b72d48247292 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7df3b53b87b3418dbc85b72d48247292. Latest consensus state: current_term: 1 leader_uuid: "7df3b53b87b3418dbc85b72d48247292" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7df3b53b87b3418dbc85b72d48247292" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34305 } } }
I20260925 22:05:10.316502 5214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7df3b53b87b3418dbc85b72d48247292 [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:10.316478 5213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7df3b53b87b3418dbc85b72d48247292 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7df3b53b87b3418dbc85b72d48247292" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7df3b53b87b3418dbc85b72d48247292" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34305 } } }
I20260925 22:05:10.316606 5213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7df3b53b87b3418dbc85b72d48247292 [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:10.316933 5221 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:05:10.317605 5221 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:05:10.319067 5221 catalog_manager.cc:1347] Generated new cluster ID: b66d9da055cf4190a35cbeb9b7fc1a17
I20260925 22:05:10.319116 5221 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:05:10 dist-test-slave-7mqr krb5kdc[5128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:10.330521 5221 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:05:10.331190 5221 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:05:10.334054 5230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:10.317526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41304 (local address 127.5.0.126:34305)
0925 22:05:10.318258 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:10.318267 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:10.318299 (+ 32us) server_negotiation.cc:407] Connection header received
0925 22:05:10.320032 (+ 1733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:10.320039 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:10.320092 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:10.320220 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:10.322252 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:10.323612 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:10.325218 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:10.325816 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:10.325914 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:10.330280 (+ 4366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:10.330286 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:10.330286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:10.330301 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:10.332185 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:10.332624 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:10.332627 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:10.332628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:10.332727 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:10.332998 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:10.332999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:10.333000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:10.333224 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:10.333377 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:10.333649 (+ 272us) server_negotiation.cc:299] Negotiation successful
0925 22:05:10.333790 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":692,"thread_start_us":65,"threads_started":1}
I20260925 22:05:10.342271 5221 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7df3b53b87b3418dbc85b72d48247292: Generated new TSK 0
I20260925 22:05:10.342465 5221 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.0.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
I20260925 22:05:10.367177 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.65:0
--local_ip_for_outbound_sockets=127.5.0.65
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:34305
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:32843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:10.448765 5235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:10.449055 5235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:10.449084 5235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:10.450192 5235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:10.450230 5235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:10.450258 5235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.65
I20260925 22:05:10.451674 5235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:32843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:34305
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.65
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:10.451869 5235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:10.452111 5235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:10.456907 5243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:10.457054 5235 server_base.cc:1034] running on GCE node
W20260925 22:05:10.456909 5241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:10.456916 5240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:10.457350 5235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:10.457603 5235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:10.458751 5235 hybrid_clock.cc:648] HybridClock initialized: now 1790373910458730 us; error 38 us; skew 500 ppm
Sep 25 22:05:10 dist-test-slave-7mqr krb5kdc[5128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373910, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:10.461205 5235 init.cc:377] Logged in from keytab as kudu/127.5.0.65@KRBTEST.COM (short username kudu)
I20260925 22:05:10.461652 5235 webserver.cc:460] Webserver started at http://127.5.0.65:42149/ using document root <none> and password file <none>
I20260925 22:05:10.461899 5235 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:10.461970 5235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:10.462117 5235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:10.463140 5235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-0/data/instance:
uuid: "b28a03897f884c88b111ca8dfe70d468"
format_stamp: "Formatted at 2026-09-25 22:05:10 on dist-test-slave-7mqr"
server_key: "0fdcdca4d6236897fc6e6ab13266a025"
server_key_iv: "5b76b580e129f8754a772de51fa6bf9c"
server_key_version: "encryptionkey@0"
I20260925 22:05:10.463526 5235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance:
uuid: "b28a03897f884c88b111ca8dfe70d468"
format_stamp: "Formatted at 2026-09-25 22:05:10 on dist-test-slave-7mqr"
server_key: "0fdcdca4d6236897fc6e6ab13266a025"
server_key_iv: "5b76b580e129f8754a772de51fa6bf9c"
server_key_version: "encryptionkey@0"
I20260925 22:05:10.464883 5235 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:05:10.465565 5250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:10.465730 5235 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:05:10.465833 5235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-0/wal
uuid: "b28a03897f884c88b111ca8dfe70d468"
format_stamp: "Formatted at 2026-09-25 22:05:10 on dist-test-slave-7mqr"
server_key: "0fdcdca4d6236897fc6e6ab13266a025"
server_key_iv: "5b76b580e129f8754a772de51fa6bf9c"
server_key_version: "encryptionkey@0"
I20260925 22:05:10.465907 5235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:10.474923 5235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:10.476903 5235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:10.477065 5235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:10.477694 5235 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:10.478006 5235 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:10.478060 5235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:10.478101 5235 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:10.478130 5235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:10.484669 5235 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.65:39681
I20260925 22:05:10.484712 5363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.65:39681 every 8 connection(s)
I20260925 22:05:10.485026 5235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
Sep 25 22:05:10 dist-test-slave-7mqr krb5kdc[5128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373910, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:10.491832 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 5235
I20260925 22:05:10.491966 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance
I20260925 22:05:10.492154 5121 external_mini_cluster.cc:1428] Setting key 25f6f68efc0942bdd644409b184c8a0f
I20260925 22:05:10.494184 5230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:10.485721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:58219 (local address 127.5.0.126:34305)
0925 22:05:10.485779 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:10.485781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:10.485954 (+ 173us) server_negotiation.cc:407] Connection header received
0925 22:05:10.486424 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:10.486425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:10.486433 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:10.486497 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:10.487390 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:10.487880 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:10.488954 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:10.489143 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:10.489194 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:10.491167 (+ 1973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:10.491168 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:10.491168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:10.491175 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:10.492701 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:10.493235 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:10.493236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:10.493240 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:10.493293 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:10.493563 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:10.493564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:10.493564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:10.493664 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:10.493713 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:10.494022 (+ 309us) server_negotiation.cc:299] Negotiation successful
0925 22:05:10.494098 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 22:05:10.494441 5366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:10.485687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34305 (local address 127.5.0.65:58219)
0925 22:05:10.485830 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:10.485870 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:10.486343 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:10.486567 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:10.486571 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:10.487131 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:10.487314 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:10.487319 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:10.488024 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:10.488025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:10.488883 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:10.488885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:10.489211 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:10.489212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:10.489305 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:10.489758 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:10.489766 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:10.491084 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:10.492793 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:10.492798 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:10.492800 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:10.493159 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:10.493358 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:10.493359 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:10.493360 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:10.493503 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:10.493725 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:10.493730 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:10.493893 (+ 163us) client_negotiation.cc:769] Sending connection context
0925 22:05:10.494004 (+ 111us) client_negotiation.cc:240] Negotiation successful
0925 22:05:10.494182 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":80,"threads_started":1}
I20260925 22:05:10.494935 5364 heartbeater.cc:347] Connected to a master server at 127.5.0.126:34305
I20260925 22:05:10.495024 5364 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:10.495213 5364 heartbeater.cc:511] Master 127.5.0.126:34305 requested a full tablet report, sending...
I20260925 22:05:10.495707 5173 ts_manager.cc:194] Registered new tserver with Master: b28a03897f884c88b111ca8dfe70d468 (127.5.0.65:39681)
I20260925 22:05:10.496366 5173 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.65@KRBTEST.COM'} at 127.5.0.65:58219
WARNING: no policy specified for kudu/127.5.0.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
I20260925 22:05:10.519305 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.66:0
--local_ip_for_outbound_sockets=127.5.0.66
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:34305
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:32843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:10.601097 5369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:10.601326 5369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:10.601365 5369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:10.602478 5369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:10.602547 5369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:10.602592 5369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.66
I20260925 22:05:10.604068 5369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:32843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:34305
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.66
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:10.604333 5369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:10.604557 5369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:10.609402 5375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:10.609411 5377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:10.609411 5374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:10.609597 5369 server_base.cc:1034] running on GCE node
I20260925 22:05:10.609856 5369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:10.610072 5369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:10.611212 5369 hybrid_clock.cc:648] HybridClock initialized: now 1790373910611200 us; error 30 us; skew 500 ppm
Sep 25 22:05:10 dist-test-slave-7mqr krb5kdc[5128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373910, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:10.613811 5369 init.cc:377] Logged in from keytab as kudu/127.5.0.66@KRBTEST.COM (short username kudu)
I20260925 22:05:10.614301 5369 webserver.cc:460] Webserver started at http://127.5.0.66:45121/ using document root <none> and password file <none>
I20260925 22:05:10.614542 5369 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:10.614609 5369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:10.614758 5369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:10.615849 5369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-1/data/instance:
uuid: "fdc94d30819245b68fabbb6f536df415"
format_stamp: "Formatted at 2026-09-25 22:05:10 on dist-test-slave-7mqr"
server_key: "350c23fecf20697e34dc3e3fe70a5248"
server_key_iv: "dfde3635bac21fff147973ccebb16aa2"
server_key_version: "encryptionkey@0"
I20260925 22:05:10.616282 5369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance:
uuid: "fdc94d30819245b68fabbb6f536df415"
format_stamp: "Formatted at 2026-09-25 22:05:10 on dist-test-slave-7mqr"
server_key: "350c23fecf20697e34dc3e3fe70a5248"
server_key_iv: "dfde3635bac21fff147973ccebb16aa2"
server_key_version: "encryptionkey@0"
I20260925 22:05:10.617763 5369 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:05:10.618426 5384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:10.618594 5369 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:05:10.618690 5369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-1/wal
uuid: "fdc94d30819245b68fabbb6f536df415"
format_stamp: "Formatted at 2026-09-25 22:05:10 on dist-test-slave-7mqr"
server_key: "350c23fecf20697e34dc3e3fe70a5248"
server_key_iv: "dfde3635bac21fff147973ccebb16aa2"
server_key_version: "encryptionkey@0"
I20260925 22:05:10.618773 5369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:10.641934 5369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:10.644384 5369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:10.644560 5369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:10.644807 5369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:10.645136 5369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:10.645191 5369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:10.645236 5369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:10.645283 5369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:10.651584 5369 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.66:41717
I20260925 22:05:10.651636 5497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.66:41717 every 8 connection(s)
I20260925 22:05:10.651968 5369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
I20260925 22:05:10.654202 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 5369
I20260925 22:05:10.654306 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance
I20260925 22:05:10.654469 5121 external_mini_cluster.cc:1428] Setting key 1f2609d4e50a43541ef61415cd207862
Sep 25 22:05:10 dist-test-slave-7mqr krb5kdc[5128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373910, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.0.67@KRBTEST.COM; defaulting to no policy
I20260925 22:05:10.662955 5230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:10.652652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:48651 (local address 127.5.0.126:34305)
0925 22:05:10.652742 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:10.652745 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:10.652934 (+ 189us) server_negotiation.cc:407] Connection header received
0925 22:05:10.653507 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:10.653508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:10.653517 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:10.653561 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:10.654818 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:10.655453 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:10.657356 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:10.657641 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:10.657728 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:10.660165 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:10.660166 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:10.660167 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:10.660175 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:10.661504 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:10.661959 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:10.661960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:10.661961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:10.662023 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:10.662303 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:10.662304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:10.662304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:10.662419 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:10.662498 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:10.662773 (+ 275us) server_negotiation.cc:299] Negotiation successful
0925 22:05:10.662861 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 22:05:10.663172 5500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:10.652572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34305 (local address 127.5.0.66:48651)
0925 22:05:10.652787 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:10.652835 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:10.653392 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:10.653623 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:10.653626 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:10.654342 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:10.654686 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:10.654692 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:10.655579 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:10.655580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:10.657269 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:10.657271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:10.657745 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:10.657747 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:10.657870 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:10.658422 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:10.658431 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:10.660072 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:10.661610 (+ 1538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:10.661616 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:10.661618 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:10.661863 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:10.662092 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:10.662093 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:10.662093 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:10.662211 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:10.662487 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:10.662494 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:10.662639 (+ 145us) client_negotiation.cc:769] Sending connection context
0925 22:05:10.662751 (+ 112us) client_negotiation.cc:240] Negotiation successful
0925 22:05:10.662913 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":102,"threads_started":1}
I20260925 22:05:10.663758 5498 heartbeater.cc:347] Connected to a master server at 127.5.0.126:34305
I20260925 22:05:10.663882 5498 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:10.664106 5498 heartbeater.cc:511] Master 127.5.0.126:34305 requested a full tablet report, sending...
I20260925 22:05:10.664541 5173 ts_manager.cc:194] Registered new tserver with Master: fdc94d30819245b68fabbb6f536df415 (127.5.0.66:41717)
I20260925 22:05:10.664937 5173 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.66@KRBTEST.COM'} at 127.5.0.66:48651
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
I20260925 22:05:10.682847 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.67:0
--local_ip_for_outbound_sockets=127.5.0.67
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:34305
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:32843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:10.764701 5503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:10.764912 5503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:10.764935 5503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:10.766095 5503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:10.766136 5503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:10.766162 5503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.67
I20260925 22:05:10.767596 5503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:32843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:34305
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.67
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:10.767779 5503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:10.767977 5503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:10.772862 5511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:10.772964 5503 server_base.cc:1034] running on GCE node
W20260925 22:05:10.772904 5508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:10.773084 5509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:10.773288 5503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:10.773540 5503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:10.774720 5503 hybrid_clock.cc:648] HybridClock initialized: now 1790373910774704 us; error 26 us; skew 500 ppm
Sep 25 22:05:10 dist-test-slave-7mqr krb5kdc[5128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373910, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:10.777170 5503 init.cc:377] Logged in from keytab as kudu/127.5.0.67@KRBTEST.COM (short username kudu)
I20260925 22:05:10.777670 5503 webserver.cc:460] Webserver started at http://127.5.0.67:39461/ using document root <none> and password file <none>
I20260925 22:05:10.777895 5503 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:10.777962 5503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:10.778113 5503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:10.779139 5503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-2/data/instance:
uuid: "ee618e4b0b3b48bc9e0d95b4bef59fa5"
format_stamp: "Formatted at 2026-09-25 22:05:10 on dist-test-slave-7mqr"
server_key: "ffe4d437f51366de65f853f930e227b7"
server_key_iv: "258aa6f92d2936b62421455de515f92e"
server_key_version: "encryptionkey@0"
I20260925 22:05:10.779502 5503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance:
uuid: "ee618e4b0b3b48bc9e0d95b4bef59fa5"
format_stamp: "Formatted at 2026-09-25 22:05:10 on dist-test-slave-7mqr"
server_key: "ffe4d437f51366de65f853f930e227b7"
server_key_iv: "258aa6f92d2936b62421455de515f92e"
server_key_version: "encryptionkey@0"
I20260925 22:05:10.780910 5503 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 22:05:10.781599 5518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:10.781782 5503 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:05:10.781884 5503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-2/wal
uuid: "ee618e4b0b3b48bc9e0d95b4bef59fa5"
format_stamp: "Formatted at 2026-09-25 22:05:10 on dist-test-slave-7mqr"
server_key: "ffe4d437f51366de65f853f930e227b7"
server_key_iv: "258aa6f92d2936b62421455de515f92e"
server_key_version: "encryptionkey@0"
I20260925 22:05:10.781956 5503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:10.792645 5503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:10.794741 5503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:10.794909 5503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:10.795166 5503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:10.795441 5503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:10.795473 5503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:10.795495 5503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:10.795507 5503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:10.802099 5503 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.67:37679
I20260925 22:05:10.802214 5631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.67:37679 every 8 connection(s)
I20260925 22:05:10.802515 5503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
I20260925 22:05:10.807693 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 5503
I20260925 22:05:10.807814 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance
I20260925 22:05:10.808034 5121 external_mini_cluster.cc:1428] Setting key d5cefe1ddf394cf44fd279d31ac80d9d
Sep 25 22:05:10 dist-test-slave-7mqr krb5kdc[5128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373910, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:10.812778 5230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:10.803292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:32801 (local address 127.5.0.126:34305)
0925 22:05:10.803376 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:10.803378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:10.803536 (+ 158us) server_negotiation.cc:407] Connection header received
0925 22:05:10.804113 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:10.804114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:10.804123 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:10.804176 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:10.805065 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:10.805563 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:10.806630 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:10.806815 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:10.806873 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:10.809838 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:10.809839 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:10.809840 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:10.809849 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:10.811394 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:10.811782 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:10.811782 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:10.811783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:10.811833 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:10.812127 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:10.812128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:10.812128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:10.812243 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:10.812297 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:10.812632 (+ 335us) server_negotiation.cc:299] Negotiation successful
0925 22:05:10.812714 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 22:05:10.812857 5634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:10.803256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34305 (local address 127.5.0.67:32801)
0925 22:05:10.803411 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:10.803446 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:10.804026 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:10.804231 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:10.804234 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:10.804800 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:10.804990 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:10.804994 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:10.805666 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:10.805667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:10.806547 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:10.806549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:10.806909 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:10.806910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:10.807021 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:10.807639 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:10.807651 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:10.809723 (+ 2072us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:10.811472 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:10.811478 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:10.811480 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:10.811712 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:10.811951 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:10.811952 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:10.811952 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:10.812066 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:10.812304 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:10.812308 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:10.812471 (+ 163us) client_negotiation.cc:769] Sending connection context
0925 22:05:10.812554 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 22:05:10.812683 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":94,"threads_started":1}
I20260925 22:05:10.813367 5632 heartbeater.cc:347] Connected to a master server at 127.5.0.126:34305
I20260925 22:05:10.813459 5632 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:10.813642 5632 heartbeater.cc:511] Master 127.5.0.126:34305 requested a full tablet report, sending...
I20260925 22:05:10.813963 5173 ts_manager.cc:194] Registered new tserver with Master: ee618e4b0b3b48bc9e0d95b4bef59fa5 (127.5.0.67:37679)
I20260925 22:05:10.814352 5173 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.67@KRBTEST.COM'} at 127.5.0.67:32801
I20260925 22:05:10.820662 5121 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 22:05:10 dist-test-slave-7mqr krb5kdc[5128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373910, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 22:05:10 dist-test-slave-7mqr krb5kdc[5128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373910, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.0.65@KRBTEST.COM
I20260925 22:05:10.845268 5639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:10.837343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54606 (local address 127.5.0.65:39681)
0925 22:05:10.837528 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:10.837533 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:10.837544 (+ 11us) server_negotiation.cc:407] Connection header received
0925 22:05:10.837572 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:10.837574 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:10.837588 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:10.837661 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:10.838210 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:10.838984 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:10.839765 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:10.840200 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:10.840267 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:10.842512 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:10.842519 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:10.842519 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:10.842526 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:10.844108 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:10.844436 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:10.844438 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:10.844439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:10.844500 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:10.844686 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:10.844686 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:10.844687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:10.844821 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:10.844939 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:10.845011 (+ 72us) server_negotiation.cc:299] Negotiation successful
0925 22:05:10.845075 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":100,"threads_started":1}
W20260925 22:05:10.845669 5278 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54606
Sep 25 22:05:10 dist-test-slave-7mqr krb5kdc[5128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 22:05:10 dist-test-slave-7mqr krb5kdc[5128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.65@KRBTEST.COM
I20260925 22:05:10.868024 5639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:10.860664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54610 (local address 127.5.0.65:39681)
0925 22:05:10.860733 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:10.860735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:10.860850 (+ 115us) server_negotiation.cc:407] Connection header received
0925 22:05:10.860927 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:10.860929 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:10.860950 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:10.861016 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:10.861602 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:10.862092 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:10.862837 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:10.863052 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:10.863129 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:10.865760 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:10.865761 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:10.865761 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:10.865768 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:10.867014 (+ 1246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:10.867322 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:10.867323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:10.867323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:10.867373 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:10.867535 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:10.867535 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:10.867536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:10.867625 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:10.867711 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:10.867777 (+ 66us) server_negotiation.cc:299] Negotiation successful
0925 22:05:10.867865 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 22:05:10.868616 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 5235
I20260925 22:05:10.873235 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 5369
I20260925 22:05:10.877991 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 5503
I20260925 22:05:10.882196 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 5141
2026-09-25T22:05:10Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1902 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:05:10 dist-test-slave-7mqr krb5kdc[5646](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:05:10 dist-test-slave-7mqr krb5kdc[5646](info): set up 2 sockets
Sep 25 22:05:10 dist-test-slave-7mqr krb5kdc[5646](info): commencing operation
krb5kdc: starting...
W20260925 22:05:11.919463 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:05:11 dist-test-slave-7mqr krb5kdc[5646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373911, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:05:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:05:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.0.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
I20260925 22:05:12.034274 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:40353
--webserver_interface=127.5.0.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:35357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:40353
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:12.115484 5659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:12.115733 5659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:12.115777 5659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:12.116922 5659 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:05:12.116992 5659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:12.117017 5659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:12.117046 5659 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:05:12.117074 5659 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:05:12.118511 5659 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:35357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:40353
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:40353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:12.118782 5659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:12.119002 5659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:12.123576 5664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:12.123680 5665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:12.123776 5659 server_base.cc:1034] running on GCE node
W20260925 22:05:12.123679 5667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:12.124082 5659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:12.124375 5659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:12.125510 5659 hybrid_clock.cc:648] HybridClock initialized: now 1790373912125498 us; error 28 us; skew 500 ppm
Sep 25 22:05:12 dist-test-slave-7mqr krb5kdc[5646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373912, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:12.127868 5659 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:05:12.128324 5659 webserver.cc:460] Webserver started at http://127.5.0.126:34631/ using document root <none> and password file <none>
I20260925 22:05:12.128549 5659 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:12.128592 5659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:12.128783 5659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:12.129856 5659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "b02f9a323888434a8109d78f5a5e80d2"
format_stamp: "Formatted at 2026-09-25 22:05:12 on dist-test-slave-7mqr"
server_key: "2146003eda8ed45a03885a38df198aec"
server_key_iv: "4027078dc78d9ae275d1667272b2bcce"
server_key_version: "encryptionkey@0"
I20260925 22:05:12.130208 5659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "b02f9a323888434a8109d78f5a5e80d2"
format_stamp: "Formatted at 2026-09-25 22:05:12 on dist-test-slave-7mqr"
server_key: "2146003eda8ed45a03885a38df198aec"
server_key_iv: "4027078dc78d9ae275d1667272b2bcce"
server_key_version: "encryptionkey@0"
I20260925 22:05:12.131575 5659 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:05:12.132361 5674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:12.132572 5659 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:12.132690 5659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "b02f9a323888434a8109d78f5a5e80d2"
format_stamp: "Formatted at 2026-09-25 22:05:12 on dist-test-slave-7mqr"
server_key: "2146003eda8ed45a03885a38df198aec"
server_key_iv: "4027078dc78d9ae275d1667272b2bcce"
server_key_version: "encryptionkey@0"
I20260925 22:05:12.132781 5659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:12.158744 5659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:12.160964 5659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:12.161140 5659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:12.165084 5659 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:40353
I20260925 22:05:12.165134 5726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:40353 every 8 connection(s)
I20260925 22:05:12.165503 5659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:05:12.166395 5727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:12.168748 5727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b02f9a323888434a8109d78f5a5e80d2: Bootstrap starting.
I20260925 22:05:12.169345 5727 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b02f9a323888434a8109d78f5a5e80d2: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:12.169646 5727 log.cc:824] T 00000000000000000000000000000000 P b02f9a323888434a8109d78f5a5e80d2: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:12.170089 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 5659
I20260925 22:05:12.170177 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:05:12.170342 5121 external_mini_cluster.cc:1428] Setting key 0b6c2a14f0a4fe7029a27012f533a0c6
I20260925 22:05:12.170410 5727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b02f9a323888434a8109d78f5a5e80d2: No bootstrap required, opened a new log
I20260925 22:05:12.172533 5727 raft_consensus.cc:348] T 00000000000000000000000000000000 P b02f9a323888434a8109d78f5a5e80d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b02f9a323888434a8109d78f5a5e80d2" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 40353 } }
I20260925 22:05:12.172703 5727 raft_consensus.cc:374] T 00000000000000000000000000000000 P b02f9a323888434a8109d78f5a5e80d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:12.172749 5727 raft_consensus.cc:729] T 00000000000000000000000000000000 P b02f9a323888434a8109d78f5a5e80d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b02f9a323888434a8109d78f5a5e80d2, State: Initialized, Role: FOLLOWER
I20260925 22:05:12.172904 5727 consensus_queue.cc:260] T 00000000000000000000000000000000 P b02f9a323888434a8109d78f5a5e80d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b02f9a323888434a8109d78f5a5e80d2" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 40353 } }
I20260925 22:05:12.173012 5727 raft_consensus.cc:388] T 00000000000000000000000000000000 P b02f9a323888434a8109d78f5a5e80d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:05:12.173056 5727 raft_consensus.cc:482] T 00000000000000000000000000000000 P b02f9a323888434a8109d78f5a5e80d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:05:12.173105 5727 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b02f9a323888434a8109d78f5a5e80d2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:12.173956 5727 raft_consensus.cc:504] T 00000000000000000000000000000000 P b02f9a323888434a8109d78f5a5e80d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b02f9a323888434a8109d78f5a5e80d2" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 40353 } }
I20260925 22:05:12.174079 5727 leader_election.cc:302] T 00000000000000000000000000000000 P b02f9a323888434a8109d78f5a5e80d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b02f9a323888434a8109d78f5a5e80d2; no voters:
I20260925 22:05:12.174279 5727 leader_election.cc:288] T 00000000000000000000000000000000 P b02f9a323888434a8109d78f5a5e80d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:05:12.174311 5732 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b02f9a323888434a8109d78f5a5e80d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:12.174412 5732 raft_consensus.cc:686] T 00000000000000000000000000000000 P b02f9a323888434a8109d78f5a5e80d2 [term 1 LEADER]: Becoming Leader. State: Replica: b02f9a323888434a8109d78f5a5e80d2, State: Running, Role: LEADER
I20260925 22:05:12.174494 5732 consensus_queue.cc:237] T 00000000000000000000000000000000 P b02f9a323888434a8109d78f5a5e80d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b02f9a323888434a8109d78f5a5e80d2" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 40353 } }
I20260925 22:05:12.174557 5727 sys_catalog.cc:564] T 00000000000000000000000000000000 P b02f9a323888434a8109d78f5a5e80d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:05:12.174810 5732 sys_catalog.cc:455] T 00000000000000000000000000000000 P b02f9a323888434a8109d78f5a5e80d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b02f9a323888434a8109d78f5a5e80d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b02f9a323888434a8109d78f5a5e80d2" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 40353 } } }
I20260925 22:05:12.174880 5732 sys_catalog.cc:458] T 00000000000000000000000000000000 P b02f9a323888434a8109d78f5a5e80d2 [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:12.175019 5732 sys_catalog.cc:455] T 00000000000000000000000000000000 P b02f9a323888434a8109d78f5a5e80d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b02f9a323888434a8109d78f5a5e80d2. Latest consensus state: current_term: 1 leader_uuid: "b02f9a323888434a8109d78f5a5e80d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b02f9a323888434a8109d78f5a5e80d2" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 40353 } } }
I20260925 22:05:12.175091 5732 sys_catalog.cc:458] T 00000000000000000000000000000000 P b02f9a323888434a8109d78f5a5e80d2 [sys.catalog]: This master's current role is: LEADER
W20260925 22:05:12.176482 5748 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b02f9a323888434a8109d78f5a5e80d2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:05:12.176541 5748 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:05:12.176595 5736 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:05:12.176751 5736 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:05:12.178412 5736 catalog_manager.cc:1347] Generated new cluster ID: 22499adadf2845d6886b2af4cff36182
I20260925 22:05:12.178464 5736 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:05:12 dist-test-slave-7mqr krb5kdc[5646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373911, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:12.183173 5731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:12.170876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36636 (local address 127.5.0.126:40353)
0925 22:05:12.171052 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:12.171062 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:12.171093 (+ 31us) server_negotiation.cc:407] Connection header received
0925 22:05:12.172504 (+ 1411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:12.172511 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:12.172560 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:12.172703 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:12.173415 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.174520 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.175923 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.176449 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.176582 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:12.179377 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:12.179382 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:12.179383 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:12.179395 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:12.181369 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:12.181758 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:12.181761 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:12.181761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:12.181886 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:12.182088 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:12.182088 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:12.182089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:12.182399 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:12.182548 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:12.182641 (+ 93us) server_negotiation.cc:299] Negotiation successful
0925 22:05:12.182968 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":76,"threads_started":1}
I20260925 22:05:12.190356 5736 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:05:12.190868 5736 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:05:12.193914 5736 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b02f9a323888434a8109d78f5a5e80d2: Generated new TSK 0
I20260925 22:05:12.194093 5736 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.0.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
I20260925 22:05:12.222865 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.65:0
--local_ip_for_outbound_sockets=127.5.0.65
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:40353
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:35357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:12.311951 5753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:12.312147 5753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:12.312173 5753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:12.313313 5753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:12.313355 5753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:12.313382 5753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.65
I20260925 22:05:12.314839 5753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:35357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:40353
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.65
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:12.315075 5753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:12.315320 5753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:12.321002 5758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:12.321002 5759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:12.321084 5761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:12.321332 5753 server_base.cc:1034] running on GCE node
I20260925 22:05:12.321514 5753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:12.321769 5753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:12.322932 5753 hybrid_clock.cc:648] HybridClock initialized: now 1790373912322912 us; error 33 us; skew 500 ppm
Sep 25 22:05:12 dist-test-slave-7mqr krb5kdc[5646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373912, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:12.325572 5753 init.cc:377] Logged in from keytab as kudu/127.5.0.65@KRBTEST.COM (short username kudu)
I20260925 22:05:12.326094 5753 webserver.cc:460] Webserver started at http://127.5.0.65:36333/ using document root <none> and password file <none>
I20260925 22:05:12.326349 5753 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:12.326421 5753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:12.326589 5753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:12.327699 5753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-0/data/instance:
uuid: "7857dc6dacdd4c85ac07c64895391c52"
format_stamp: "Formatted at 2026-09-25 22:05:12 on dist-test-slave-7mqr"
server_key: "24f8c65562bd9cb65c2f6ba9749b0080"
server_key_iv: "0a43879d7d708e1adfbbc541a2d53edf"
server_key_version: "encryptionkey@0"
I20260925 22:05:12.328080 5753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance:
uuid: "7857dc6dacdd4c85ac07c64895391c52"
format_stamp: "Formatted at 2026-09-25 22:05:12 on dist-test-slave-7mqr"
server_key: "24f8c65562bd9cb65c2f6ba9749b0080"
server_key_iv: "0a43879d7d708e1adfbbc541a2d53edf"
server_key_version: "encryptionkey@0"
I20260925 22:05:12.329394 5753 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:05:12.330171 5768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:12.330358 5753 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:05:12.330456 5753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-0/wal
uuid: "7857dc6dacdd4c85ac07c64895391c52"
format_stamp: "Formatted at 2026-09-25 22:05:12 on dist-test-slave-7mqr"
server_key: "24f8c65562bd9cb65c2f6ba9749b0080"
server_key_iv: "0a43879d7d708e1adfbbc541a2d53edf"
server_key_version: "encryptionkey@0"
I20260925 22:05:12.330538 5753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:12.358186 5753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:12.360090 5753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:12.360263 5753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:12.360505 5753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:12.360831 5753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:12.360885 5753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:12.360926 5753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:12.360955 5753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:12.366765 5753 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.65:35251
I20260925 22:05:12.366811 5881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.65:35251 every 8 connection(s)
I20260925 22:05:12.367133 5753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
I20260925 22:05:12.368196 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 5753
I20260925 22:05:12.368315 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance
I20260925 22:05:12.368484 5121 external_mini_cluster.cc:1428] Setting key 0ed2ec7f4897b69c760541835eb12aaa
Sep 25 22:05:12 dist-test-slave-7mqr krb5kdc[5646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373912, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.0.66@KRBTEST.COM; defaulting to no policy
I20260925 22:05:12.377969 5731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:12.367960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:45931 (local address 127.5.0.126:40353)
0925 22:05:12.368015 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:12.368018 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:12.368201 (+ 183us) server_negotiation.cc:407] Connection header received
0925 22:05:12.369287 (+ 1086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:12.369288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:12.369302 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:12.369355 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:12.369994 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.370814 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.372318 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.372517 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.372598 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:12.375158 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:12.375159 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:12.375159 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:12.375166 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:12.376628 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:12.377114 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:12.377115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:12.377115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:12.377168 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:12.377447 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:12.377447 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:12.377448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:12.377544 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:12.377593 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:12.377842 (+ 249us) server_negotiation.cc:299] Negotiation successful
0925 22:05:12.377905 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 22:05:12.378139 5884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:12.367820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:40353 (local address 127.5.0.65:45931)
0925 22:05:12.368055 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:12.368110 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:12.369177 (+ 1067us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:12.369440 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:12.369444 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:12.369706 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:12.369892 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:12.369896 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.370923 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.370924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:12.372234 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:12.372237 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.372621 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.372622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:12.372747 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:12.373297 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:12.373305 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:12.375067 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:12.376716 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:12.376730 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:12.376733 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:12.377041 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:12.377238 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:12.377239 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:12.377240 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:12.377386 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:12.377601 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:12.377606 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:12.377720 (+ 114us) client_negotiation.cc:769] Sending connection context
0925 22:05:12.377816 (+ 96us) client_negotiation.cc:240] Negotiation successful
0925 22:05:12.377933 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":111,"threads_started":1}
I20260925 22:05:12.378684 5882 heartbeater.cc:347] Connected to a master server at 127.5.0.126:40353
I20260925 22:05:12.378770 5882 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:12.378907 5882 heartbeater.cc:511] Master 127.5.0.126:40353 requested a full tablet report, sending...
I20260925 22:05:12.379420 5691 ts_manager.cc:194] Registered new tserver with Master: 7857dc6dacdd4c85ac07c64895391c52 (127.5.0.65:35251)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.66@KRBTEST.COM" created.
I20260925 22:05:12.380267 5691 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.65@KRBTEST.COM'} at 127.5.0.65:45931
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
I20260925 22:05:12.394068 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.66:0
--local_ip_for_outbound_sockets=127.5.0.66
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:40353
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:35357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:12.475488 5887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:12.475688 5887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:12.475711 5887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:12.476920 5887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:12.476964 5887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:12.476991 5887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.66
I20260925 22:05:12.478581 5887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:35357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:40353
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.66
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:12.478834 5887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:12.479064 5887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:12.483996 5895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:12.483989 5893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:12.484017 5887 server_base.cc:1034] running on GCE node
W20260925 22:05:12.483990 5892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:12.484372 5887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:12.484601 5887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:12.485741 5887 hybrid_clock.cc:648] HybridClock initialized: now 1790373912485732 us; error 32 us; skew 500 ppm
Sep 25 22:05:12 dist-test-slave-7mqr krb5kdc[5646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373912, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:12.488152 5887 init.cc:377] Logged in from keytab as kudu/127.5.0.66@KRBTEST.COM (short username kudu)
I20260925 22:05:12.488565 5887 webserver.cc:460] Webserver started at http://127.5.0.66:43583/ using document root <none> and password file <none>
I20260925 22:05:12.488799 5887 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:12.488867 5887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:12.489014 5887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:12.490025 5887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-1/data/instance:
uuid: "8e5a0f52d78d4f9c9c1fe750f04d8c23"
format_stamp: "Formatted at 2026-09-25 22:05:12 on dist-test-slave-7mqr"
server_key: "c93b9270059c79ecc8a3108e1c505022"
server_key_iv: "c70cb95b941d982df8f6e378dce96244"
server_key_version: "encryptionkey@0"
I20260925 22:05:12.490386 5887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance:
uuid: "8e5a0f52d78d4f9c9c1fe750f04d8c23"
format_stamp: "Formatted at 2026-09-25 22:05:12 on dist-test-slave-7mqr"
server_key: "c93b9270059c79ecc8a3108e1c505022"
server_key_iv: "c70cb95b941d982df8f6e378dce96244"
server_key_version: "encryptionkey@0"
I20260925 22:05:12.491689 5887 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:05:12.492492 5902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:12.492713 5887 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 22:05:12.492808 5887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-1/wal
uuid: "8e5a0f52d78d4f9c9c1fe750f04d8c23"
format_stamp: "Formatted at 2026-09-25 22:05:12 on dist-test-slave-7mqr"
server_key: "c93b9270059c79ecc8a3108e1c505022"
server_key_iv: "c70cb95b941d982df8f6e378dce96244"
server_key_version: "encryptionkey@0"
I20260925 22:05:12.492892 5887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:12.528892 5887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:12.530841 5887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:12.531003 5887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:12.531252 5887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:12.531553 5887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:12.531608 5887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:12.531643 5887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:12.531695 5887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:12.537825 5887 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.66:44489
I20260925 22:05:12.537873 6015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.66:44489 every 8 connection(s)
I20260925 22:05:12.538237 5887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
I20260925 22:05:12.538841 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 5887
I20260925 22:05:12.538946 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance
I20260925 22:05:12.539124 5121 external_mini_cluster.cc:1428] Setting key e311b85a2fb653c6e2893aa4367a7a08
Sep 25 22:05:12 dist-test-slave-7mqr krb5kdc[5646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373912, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.0.67@KRBTEST.COM; defaulting to no policy
I20260925 22:05:12.549708 5731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:12.539323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:36441 (local address 127.5.0.126:40353)
0925 22:05:12.539399 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:12.539402 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:12.539412 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:05:12.540611 (+ 1199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:12.540612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:12.540623 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:12.540675 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:12.541583 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.542367 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.543994 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.544189 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.544240 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:12.546705 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:12.546707 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:12.546707 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:12.546715 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:12.548282 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:12.548717 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:12.548718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:12.548718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:12.548781 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:12.549034 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:12.549035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:12.549035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:12.549153 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:12.549228 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:12.549509 (+ 281us) server_negotiation.cc:299] Negotiation successful
0925 22:05:12.549593 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 22:05:12.549885 6018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:12.538992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:40353 (local address 127.5.0.66:36441)
0925 22:05:12.539163 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:12.539216 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:12.540503 (+ 1287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:12.540763 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:12.540767 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:12.541176 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:12.541481 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:12.541487 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.542479 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.542480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:12.543878 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:12.543881 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.544282 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.544283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:12.544434 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:12.544963 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:12.544975 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:12.546607 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:12.548378 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:12.548385 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:12.548387 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:12.548626 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:12.548848 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:12.548849 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:12.548850 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:12.548958 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:12.549218 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:12.549224 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:12.549373 (+ 149us) client_negotiation.cc:769] Sending connection context
0925 22:05:12.549488 (+ 115us) client_negotiation.cc:240] Negotiation successful
0925 22:05:12.549649 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":81,"threads_started":1}
I20260925 22:05:12.550506 6016 heartbeater.cc:347] Connected to a master server at 127.5.0.126:40353
I20260925 22:05:12.550607 6016 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:12.550836 6016 heartbeater.cc:511] Master 127.5.0.126:40353 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.67@KRBTEST.COM" created.
I20260925 22:05:12.551278 5691 ts_manager.cc:194] Registered new tserver with Master: 8e5a0f52d78d4f9c9c1fe750f04d8c23 (127.5.0.66:44489)
I20260925 22:05:12.551659 5691 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.66@KRBTEST.COM'} at 127.5.0.66:36441
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
I20260925 22:05:12.566562 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.67:0
--local_ip_for_outbound_sockets=127.5.0.67
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:40353
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:35357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:12.647008 6021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:12.647198 6021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:12.647221 6021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:12.648419 6021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:12.648459 6021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:12.648507 6021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.67
I20260925 22:05:12.649971 6021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:35357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:40353
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.67
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:12.650207 6021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:12.650442 6021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:12.655308 6027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:12.655325 6021 server_base.cc:1034] running on GCE node
W20260925 22:05:12.655335 6026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:12.655309 6029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:12.655689 6021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:12.655997 6021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:12.657153 6021 hybrid_clock.cc:648] HybridClock initialized: now 1790373912657145 us; error 20 us; skew 500 ppm
Sep 25 22:05:12 dist-test-slave-7mqr krb5kdc[5646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373912, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:12.659521 6021 init.cc:377] Logged in from keytab as kudu/127.5.0.67@KRBTEST.COM (short username kudu)
I20260925 22:05:12.660059 6021 webserver.cc:460] Webserver started at http://127.5.0.67:35459/ using document root <none> and password file <none>
I20260925 22:05:12.660295 6021 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:12.660360 6021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:12.660498 6021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:12.661538 6021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-2/data/instance:
uuid: "1100629d47e24c089b043f3f70597104"
format_stamp: "Formatted at 2026-09-25 22:05:12 on dist-test-slave-7mqr"
server_key: "1922d38c6ca83a0aade94326007c211f"
server_key_iv: "864f80929cc03f21037c20a5f6bf2f65"
server_key_version: "encryptionkey@0"
I20260925 22:05:12.661902 6021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance:
uuid: "1100629d47e24c089b043f3f70597104"
format_stamp: "Formatted at 2026-09-25 22:05:12 on dist-test-slave-7mqr"
server_key: "1922d38c6ca83a0aade94326007c211f"
server_key_iv: "864f80929cc03f21037c20a5f6bf2f65"
server_key_version: "encryptionkey@0"
I20260925 22:05:12.663192 6021 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:05:12.663961 6036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:12.664130 6021 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:05:12.664230 6021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-2/wal
uuid: "1100629d47e24c089b043f3f70597104"
format_stamp: "Formatted at 2026-09-25 22:05:12 on dist-test-slave-7mqr"
server_key: "1922d38c6ca83a0aade94326007c211f"
server_key_iv: "864f80929cc03f21037c20a5f6bf2f65"
server_key_version: "encryptionkey@0"
I20260925 22:05:12.664317 6021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:12.682281 6021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:12.684556 6021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:12.684719 6021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:12.684958 6021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:12.685285 6021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:12.685339 6021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:12.685384 6021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:12.685411 6021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:12.690928 6021 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.67:37497
I20260925 22:05:12.691021 6149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.67:37497 every 8 connection(s)
I20260925 22:05:12.691340 6021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
Sep 25 22:05:12 dist-test-slave-7mqr krb5kdc[5646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373912, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:12.700788 5731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:12.692162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:35511 (local address 127.5.0.126:40353)
0925 22:05:12.692206 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:12.692208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:12.692400 (+ 192us) server_negotiation.cc:407] Connection header received
0925 22:05:12.693383 (+ 983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:12.693384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:12.693398 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:12.693459 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:12.694168 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.694681 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.695789 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.696071 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.696155 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:12.698211 (+ 2056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:12.698212 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:12.698213 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:12.698221 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:12.699411 (+ 1190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:12.699830 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:12.699831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:12.699832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:12.699906 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:12.700146 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:12.700147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:12.700147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:12.700265 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:12.700341 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:12.700614 (+ 273us) server_negotiation.cc:299] Negotiation successful
0925 22:05:12.700693 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 22:05:12.700994 6152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:12.692129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:40353 (local address 127.5.0.67:35511)
0925 22:05:12.692269 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:12.692315 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:12.693258 (+ 943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:12.693535 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:12.693540 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:12.693817 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:12.694073 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:12.694078 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.694777 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.694779 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:12.695704 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:12.695706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.696163 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.696164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:12.696291 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:12.696772 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:12.696781 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:12.698117 (+ 1336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:12.699494 (+ 1377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:12.699500 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:12.699502 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:12.699751 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:12.699966 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:12.699967 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:12.699967 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:12.700068 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:12.700330 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:12.700336 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:12.700471 (+ 135us) client_negotiation.cc:769] Sending connection context
0925 22:05:12.700586 (+ 115us) client_negotiation.cc:240] Negotiation successful
0925 22:05:12.700757 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":71,"threads_started":1}
I20260925 22:05:12.701272 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 6021
I20260925 22:05:12.701363 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance
I20260925 22:05:12.701547 5121 external_mini_cluster.cc:1428] Setting key 3308f9a64682102087c3690c2a560b35
I20260925 22:05:12.701545 6150 heartbeater.cc:347] Connected to a master server at 127.5.0.126:40353
I20260925 22:05:12.701643 6150 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:12.701853 6150 heartbeater.cc:511] Master 127.5.0.126:40353 requested a full tablet report, sending...
I20260925 22:05:12.702260 5690 ts_manager.cc:194] Registered new tserver with Master: 1100629d47e24c089b043f3f70597104 (127.5.0.67:37497)
I20260925 22:05:12.702641 5690 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.67@KRBTEST.COM'} at 127.5.0.67:35511
I20260925 22:05:12.713229 5121 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:05:12.720957 5731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:12.714904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47308 (local address 127.5.0.126:40353)
0925 22:05:12.714975 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:12.714977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:12.715067 (+ 90us) server_negotiation.cc:407] Connection header received
0925 22:05:12.715157 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:12.715158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:12.715169 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:12.715244 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:12.715838 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.716393 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.717124 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.717326 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.717413 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:12.718579 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:12.718581 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:12.718581 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:12.718589 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:12.719698 (+ 1109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:12.720141 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:12.720143 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:12.720143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:12.720203 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:12.720405 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:12.720406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:12.720406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:12.720560 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:12.720669 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:12.720771 (+ 102us) server_negotiation.cc:299] Negotiation successful
0925 22:05:12.720852 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 22:05:12.723281 5690 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47308:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:05:12.724926 5690 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:05:12.731364 6161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:12.727189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:35251 (local address 127.0.0.1:54982)
0925 22:05:12.727422 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:12.727433 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:12.727487 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:12.727971 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:12.727972 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:12.727974 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:12.728028 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:12.728030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.729334 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.729335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:12.730404 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:12.730406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.731066 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.731066 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:12.731146 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:12.731159 (+ 13us) client_negotiation.cc:769] Sending connection context
0925 22:05:12.731234 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 22:05:12.731288 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":108,"threads_started":1}
I20260925 22:05:12.731377 6166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:12.727562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54982 (local address 127.5.0.65:35251)
0925 22:05:12.727743 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:12.727747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:12.727756 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:05:12.727790 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:12.727792 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:12.727806 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:12.727875 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:12.728123 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.729224 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.730487 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.730978 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.731066 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:12.731127 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:12.731267 (+ 140us) server_negotiation.cc:299] Negotiation successful
0925 22:05:12.731307 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":90,"threads_started":1}
I20260925 22:05:12.731506 6165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:12.727421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43736 (local address 127.5.0.67:37497)
0925 22:05:12.727606 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:12.727610 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:12.727620 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:05:12.727732 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:12.727735 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:12.727753 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:12.727840 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:12.728055 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.728843 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.730315 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.731040 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.731109 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:12.731163 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:12.731384 (+ 221us) server_negotiation.cc:299] Negotiation successful
0925 22:05:12.731426 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":76,"threads_started":1}
I20260925 22:05:12.731639 6164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:12.727369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52310 (local address 127.5.0.66:44489)
0925 22:05:12.727771 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:12.727775 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:12.727783 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:05:12.727834 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:12.727836 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:12.727853 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:12.728056 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:12.728292 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.729305 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.730379 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.731040 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.731109 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:12.731166 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:12.731526 (+ 360us) server_negotiation.cc:299] Negotiation successful
0925 22:05:12.731570 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":81,"threads_started":1}
I20260925 22:05:12.731459 6163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:12.727172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:44489 (local address 127.0.0.1:52310)
0925 22:05:12.727360 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:12.727403 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:12.727471 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:12.728014 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:12.728015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:12.728017 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:12.728210 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:12.728211 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.729411 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.729412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:12.730292 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:12.730294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.731203 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.731204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:12.731287 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:12.731291 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:05:12.731343 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 22:05:12.731388 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":22,"thread_start_us":46,"threads_started":1}
I20260925 22:05:12.731698 6162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:12.727200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:37497 (local address 127.0.0.1:43736)
0925 22:05:12.727573 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:12.727583 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:12.727622 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:12.727860 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:12.727863 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:12.727866 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:12.727960 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:12.727962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.728939 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.728939 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:12.730231 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:12.730233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.731201 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.731203 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:12.731292 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:12.731295 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:05:12.731578 (+ 283us) client_negotiation.cc:240] Negotiation successful
0925 22:05:12.731627 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":2,"thread_start_us":94,"threads_started":1}
I20260925 22:05:12.732698 5816 tablet_service.cc:1467] Processing CreateTablet for tablet 5210bf68d0df44c9b70bc3b5308b12f7 (DEFAULT_TABLE table=test-table [id=e1a4c0ce10904e5dbcc1677759c1a353]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:12.732720 5950 tablet_service.cc:1467] Processing CreateTablet for tablet 5210bf68d0df44c9b70bc3b5308b12f7 (DEFAULT_TABLE table=test-table [id=e1a4c0ce10904e5dbcc1677759c1a353]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:12.732733 6084 tablet_service.cc:1467] Processing CreateTablet for tablet 5210bf68d0df44c9b70bc3b5308b12f7 (DEFAULT_TABLE table=test-table [id=e1a4c0ce10904e5dbcc1677759c1a353]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:12.733045 5816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5210bf68d0df44c9b70bc3b5308b12f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:12.733044 5950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5210bf68d0df44c9b70bc3b5308b12f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:12.733044 6084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5210bf68d0df44c9b70bc3b5308b12f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:12.736073 6167 tablet_bootstrap.cc:492] T 5210bf68d0df44c9b70bc3b5308b12f7 P 7857dc6dacdd4c85ac07c64895391c52: Bootstrap starting.
I20260925 22:05:12.736244 6169 tablet_bootstrap.cc:492] T 5210bf68d0df44c9b70bc3b5308b12f7 P 1100629d47e24c089b043f3f70597104: Bootstrap starting.
I20260925 22:05:12.736930 6167 tablet_bootstrap.cc:654] T 5210bf68d0df44c9b70bc3b5308b12f7 P 7857dc6dacdd4c85ac07c64895391c52: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:12.736930 6169 tablet_bootstrap.cc:654] T 5210bf68d0df44c9b70bc3b5308b12f7 P 1100629d47e24c089b043f3f70597104: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:12.737571 6169 log.cc:824] T 5210bf68d0df44c9b70bc3b5308b12f7 P 1100629d47e24c089b043f3f70597104: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:12.737560 6167 log.cc:824] T 5210bf68d0df44c9b70bc3b5308b12f7 P 7857dc6dacdd4c85ac07c64895391c52: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:12.737893 6168 tablet_bootstrap.cc:492] T 5210bf68d0df44c9b70bc3b5308b12f7 P 8e5a0f52d78d4f9c9c1fe750f04d8c23: Bootstrap starting.
I20260925 22:05:12.738255 6169 tablet_bootstrap.cc:492] T 5210bf68d0df44c9b70bc3b5308b12f7 P 1100629d47e24c089b043f3f70597104: No bootstrap required, opened a new log
I20260925 22:05:12.738338 6169 ts_tablet_manager.cc:1397] T 5210bf68d0df44c9b70bc3b5308b12f7 P 1100629d47e24c089b043f3f70597104: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:05:12.738471 6168 tablet_bootstrap.cc:654] T 5210bf68d0df44c9b70bc3b5308b12f7 P 8e5a0f52d78d4f9c9c1fe750f04d8c23: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:12.739044 6168 log.cc:824] T 5210bf68d0df44c9b70bc3b5308b12f7 P 8e5a0f52d78d4f9c9c1fe750f04d8c23: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:12.739675 6168 tablet_bootstrap.cc:492] T 5210bf68d0df44c9b70bc3b5308b12f7 P 8e5a0f52d78d4f9c9c1fe750f04d8c23: No bootstrap required, opened a new log
I20260925 22:05:12.739734 6168 ts_tablet_manager.cc:1397] T 5210bf68d0df44c9b70bc3b5308b12f7 P 8e5a0f52d78d4f9c9c1fe750f04d8c23: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:05:12.740038 6169 raft_consensus.cc:348] T 5210bf68d0df44c9b70bc3b5308b12f7 P 1100629d47e24c089b043f3f70597104 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5a0f52d78d4f9c9c1fe750f04d8c23" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44489 } } peers { permanent_uuid: "7857dc6dacdd4c85ac07c64895391c52" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 35251 } } peers { permanent_uuid: "1100629d47e24c089b043f3f70597104" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37497 } }
I20260925 22:05:12.740222 6169 raft_consensus.cc:374] T 5210bf68d0df44c9b70bc3b5308b12f7 P 1100629d47e24c089b043f3f70597104 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:12.740268 6169 raft_consensus.cc:729] T 5210bf68d0df44c9b70bc3b5308b12f7 P 1100629d47e24c089b043f3f70597104 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1100629d47e24c089b043f3f70597104, State: Initialized, Role: FOLLOWER
I20260925 22:05:12.740382 6169 consensus_queue.cc:260] T 5210bf68d0df44c9b70bc3b5308b12f7 P 1100629d47e24c089b043f3f70597104 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5a0f52d78d4f9c9c1fe750f04d8c23" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44489 } } peers { permanent_uuid: "7857dc6dacdd4c85ac07c64895391c52" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 35251 } } peers { permanent_uuid: "1100629d47e24c089b043f3f70597104" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37497 } }
I20260925 22:05:12.740664 6169 ts_tablet_manager.cc:1428] T 5210bf68d0df44c9b70bc3b5308b12f7 P 1100629d47e24c089b043f3f70597104: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260925 22:05:12.740773 6167 tablet_bootstrap.cc:492] T 5210bf68d0df44c9b70bc3b5308b12f7 P 7857dc6dacdd4c85ac07c64895391c52: No bootstrap required, opened a new log
I20260925 22:05:12.740808 6150 heartbeater.cc:503] Master 127.5.0.126:40353 was elected leader, sending a full tablet report...
I20260925 22:05:12.740839 6167 ts_tablet_manager.cc:1397] T 5210bf68d0df44c9b70bc3b5308b12f7 P 7857dc6dacdd4c85ac07c64895391c52: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260925 22:05:12.742194 6168 raft_consensus.cc:348] T 5210bf68d0df44c9b70bc3b5308b12f7 P 8e5a0f52d78d4f9c9c1fe750f04d8c23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5a0f52d78d4f9c9c1fe750f04d8c23" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44489 } } peers { permanent_uuid: "7857dc6dacdd4c85ac07c64895391c52" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 35251 } } peers { permanent_uuid: "1100629d47e24c089b043f3f70597104" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37497 } }
I20260925 22:05:12.742376 6168 raft_consensus.cc:374] T 5210bf68d0df44c9b70bc3b5308b12f7 P 8e5a0f52d78d4f9c9c1fe750f04d8c23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:12.742411 6168 raft_consensus.cc:729] T 5210bf68d0df44c9b70bc3b5308b12f7 P 8e5a0f52d78d4f9c9c1fe750f04d8c23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e5a0f52d78d4f9c9c1fe750f04d8c23, State: Initialized, Role: FOLLOWER
I20260925 22:05:12.742527 6168 consensus_queue.cc:260] T 5210bf68d0df44c9b70bc3b5308b12f7 P 8e5a0f52d78d4f9c9c1fe750f04d8c23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5a0f52d78d4f9c9c1fe750f04d8c23" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44489 } } peers { permanent_uuid: "7857dc6dacdd4c85ac07c64895391c52" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 35251 } } peers { permanent_uuid: "1100629d47e24c089b043f3f70597104" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37497 } }
I20260925 22:05:12.742818 6016 heartbeater.cc:503] Master 127.5.0.126:40353 was elected leader, sending a full tablet report...
I20260925 22:05:12.742815 6168 ts_tablet_manager.cc:1428] T 5210bf68d0df44c9b70bc3b5308b12f7 P 8e5a0f52d78d4f9c9c1fe750f04d8c23: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 22:05:12.742869 6167 raft_consensus.cc:348] T 5210bf68d0df44c9b70bc3b5308b12f7 P 7857dc6dacdd4c85ac07c64895391c52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5a0f52d78d4f9c9c1fe750f04d8c23" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44489 } } peers { permanent_uuid: "7857dc6dacdd4c85ac07c64895391c52" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 35251 } } peers { permanent_uuid: "1100629d47e24c089b043f3f70597104" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37497 } }
I20260925 22:05:12.743032 6167 raft_consensus.cc:374] T 5210bf68d0df44c9b70bc3b5308b12f7 P 7857dc6dacdd4c85ac07c64895391c52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:12.743080 6167 raft_consensus.cc:729] T 5210bf68d0df44c9b70bc3b5308b12f7 P 7857dc6dacdd4c85ac07c64895391c52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7857dc6dacdd4c85ac07c64895391c52, State: Initialized, Role: FOLLOWER
I20260925 22:05:12.743198 6167 consensus_queue.cc:260] T 5210bf68d0df44c9b70bc3b5308b12f7 P 7857dc6dacdd4c85ac07c64895391c52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5a0f52d78d4f9c9c1fe750f04d8c23" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44489 } } peers { permanent_uuid: "7857dc6dacdd4c85ac07c64895391c52" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 35251 } } peers { permanent_uuid: "1100629d47e24c089b043f3f70597104" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37497 } }
I20260925 22:05:12.743502 6167 ts_tablet_manager.cc:1428] T 5210bf68d0df44c9b70bc3b5308b12f7 P 7857dc6dacdd4c85ac07c64895391c52: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 22:05:12.743506 5882 heartbeater.cc:503] Master 127.5.0.126:40353 was elected leader, sending a full tablet report...
W20260925 22:05:12.788812 6017 tablet.cc:2369] T 5210bf68d0df44c9b70bc3b5308b12f7 P 8e5a0f52d78d4f9c9c1fe750f04d8c23: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 22:05:12.867775 5883 tablet.cc:2369] T 5210bf68d0df44c9b70bc3b5308b12f7 P 7857dc6dacdd4c85ac07c64895391c52: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:05:12.923807 6175 raft_consensus.cc:482] T 5210bf68d0df44c9b70bc3b5308b12f7 P 7857dc6dacdd4c85ac07c64895391c52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:05:12.923997 6175 raft_consensus.cc:504] T 5210bf68d0df44c9b70bc3b5308b12f7 P 7857dc6dacdd4c85ac07c64895391c52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5a0f52d78d4f9c9c1fe750f04d8c23" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44489 } } peers { permanent_uuid: "7857dc6dacdd4c85ac07c64895391c52" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 35251 } } peers { permanent_uuid: "1100629d47e24c089b043f3f70597104" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37497 } }
I20260925 22:05:12.924422 6175 leader_election.cc:288] T 5210bf68d0df44c9b70bc3b5308b12f7 P 7857dc6dacdd4c85ac07c64895391c52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e5a0f52d78d4f9c9c1fe750f04d8c23 (127.5.0.66:44489), 1100629d47e24c089b043f3f70597104 (127.5.0.67:37497)
I20260925 22:05:12.928548 6164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:12.924681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:42145 (local address 127.5.0.66:44489)
0925 22:05:12.924742 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:12.924745 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:12.924859 (+ 114us) server_negotiation.cc:407] Connection header received
0925 22:05:12.924980 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:12.924981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:12.924996 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:12.925057 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:12.925278 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.926025 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.927577 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.928175 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.928252 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:12.928263 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:12.928436 (+ 173us) server_negotiation.cc:299] Negotiation successful
0925 22:05:12.928475 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 22:05:12.928548 6165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:12.924715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:38019 (local address 127.5.0.67:37497)
0925 22:05:12.924758 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:12.924761 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:12.924973 (+ 212us) server_negotiation.cc:407] Connection header received
0925 22:05:12.925020 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:12.925021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:12.925032 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:12.925079 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:12.925278 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.926014 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.927577 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.928179 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.928252 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:12.928263 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:12.928437 (+ 174us) server_negotiation.cc:299] Negotiation successful
0925 22:05:12.928475 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 22:05:12.928591 6176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:12.924649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:37497 (local address 127.5.0.65:38019)
0925 22:05:12.924901 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:12.924910 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:12.924960 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:12.925089 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:12.925090 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:12.925094 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:12.925195 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:12.925197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.926104 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.926105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:12.927486 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:12.927489 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.928273 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.928273 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:12.928353 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:12.928359 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 22:05:12.928419 (+ 60us) client_negotiation.cc:240] Negotiation successful
0925 22:05:12.928479 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":82,"thread_start_us":68,"threads_started":1}
I20260925 22:05:12.928591 6177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:12.924610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:44489 (local address 127.5.0.65:42145)
0925 22:05:12.924786 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:12.924797 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:12.924887 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:12.925089 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:12.925090 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:12.925093 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:12.925201 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:12.925203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.926105 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.926105 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:12.927486 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:12.927489 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.928271 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.928272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:12.928353 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:12.928357 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:05:12.928419 (+ 62us) client_negotiation.cc:240] Negotiation successful
0925 22:05:12.928479 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":106,"thread_start_us":87,"threads_started":1}
I20260925 22:05:12.928953 6104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5210bf68d0df44c9b70bc3b5308b12f7" candidate_uuid: "7857dc6dacdd4c85ac07c64895391c52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1100629d47e24c089b043f3f70597104" is_pre_election: true
I20260925 22:05:12.929121 6104 raft_consensus.cc:2454] T 5210bf68d0df44c9b70bc3b5308b12f7 P 1100629d47e24c089b043f3f70597104 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7857dc6dacdd4c85ac07c64895391c52 in term 0.
I20260925 22:05:12.929119 5970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5210bf68d0df44c9b70bc3b5308b12f7" candidate_uuid: "7857dc6dacdd4c85ac07c64895391c52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e5a0f52d78d4f9c9c1fe750f04d8c23" is_pre_election: true
I20260925 22:05:12.929246 5970 raft_consensus.cc:2454] T 5210bf68d0df44c9b70bc3b5308b12f7 P 8e5a0f52d78d4f9c9c1fe750f04d8c23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7857dc6dacdd4c85ac07c64895391c52 in term 0.
I20260925 22:05:12.929386 5770 leader_election.cc:302] T 5210bf68d0df44c9b70bc3b5308b12f7 P 7857dc6dacdd4c85ac07c64895391c52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1100629d47e24c089b043f3f70597104, 7857dc6dacdd4c85ac07c64895391c52; no voters:
I20260925 22:05:12.929551 6175 raft_consensus.cc:2781] T 5210bf68d0df44c9b70bc3b5308b12f7 P 7857dc6dacdd4c85ac07c64895391c52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:05:12.929625 6175 raft_consensus.cc:482] T 5210bf68d0df44c9b70bc3b5308b12f7 P 7857dc6dacdd4c85ac07c64895391c52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:05:12.929642 6175 raft_consensus.cc:3037] T 5210bf68d0df44c9b70bc3b5308b12f7 P 7857dc6dacdd4c85ac07c64895391c52 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:12.930279 6175 raft_consensus.cc:504] T 5210bf68d0df44c9b70bc3b5308b12f7 P 7857dc6dacdd4c85ac07c64895391c52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5a0f52d78d4f9c9c1fe750f04d8c23" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44489 } } peers { permanent_uuid: "7857dc6dacdd4c85ac07c64895391c52" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 35251 } } peers { permanent_uuid: "1100629d47e24c089b043f3f70597104" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37497 } }
I20260925 22:05:12.930418 6175 leader_election.cc:288] T 5210bf68d0df44c9b70bc3b5308b12f7 P 7857dc6dacdd4c85ac07c64895391c52 [CANDIDATE]: Term 1 election: Requested vote from peers 8e5a0f52d78d4f9c9c1fe750f04d8c23 (127.5.0.66:44489), 1100629d47e24c089b043f3f70597104 (127.5.0.67:37497)
I20260925 22:05:12.930599 5970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5210bf68d0df44c9b70bc3b5308b12f7" candidate_uuid: "7857dc6dacdd4c85ac07c64895391c52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e5a0f52d78d4f9c9c1fe750f04d8c23"
I20260925 22:05:12.930601 6104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5210bf68d0df44c9b70bc3b5308b12f7" candidate_uuid: "7857dc6dacdd4c85ac07c64895391c52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1100629d47e24c089b043f3f70597104"
I20260925 22:05:12.930675 5970 raft_consensus.cc:3037] T 5210bf68d0df44c9b70bc3b5308b12f7 P 8e5a0f52d78d4f9c9c1fe750f04d8c23 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:12.930675 6104 raft_consensus.cc:3037] T 5210bf68d0df44c9b70bc3b5308b12f7 P 1100629d47e24c089b043f3f70597104 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:12.931519 5970 raft_consensus.cc:2454] T 5210bf68d0df44c9b70bc3b5308b12f7 P 8e5a0f52d78d4f9c9c1fe750f04d8c23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7857dc6dacdd4c85ac07c64895391c52 in term 1.
I20260925 22:05:12.931519 6104 raft_consensus.cc:2454] T 5210bf68d0df44c9b70bc3b5308b12f7 P 1100629d47e24c089b043f3f70597104 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7857dc6dacdd4c85ac07c64895391c52 in term 1.
I20260925 22:05:12.931687 5770 leader_election.cc:302] T 5210bf68d0df44c9b70bc3b5308b12f7 P 7857dc6dacdd4c85ac07c64895391c52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1100629d47e24c089b043f3f70597104, 7857dc6dacdd4c85ac07c64895391c52; no voters:
I20260925 22:05:12.931813 6175 raft_consensus.cc:2781] T 5210bf68d0df44c9b70bc3b5308b12f7 P 7857dc6dacdd4c85ac07c64895391c52 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:12.932016 6175 raft_consensus.cc:686] T 5210bf68d0df44c9b70bc3b5308b12f7 P 7857dc6dacdd4c85ac07c64895391c52 [term 1 LEADER]: Becoming Leader. State: Replica: 7857dc6dacdd4c85ac07c64895391c52, State: Running, Role: LEADER
I20260925 22:05:12.932121 6175 consensus_queue.cc:237] T 5210bf68d0df44c9b70bc3b5308b12f7 P 7857dc6dacdd4c85ac07c64895391c52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5a0f52d78d4f9c9c1fe750f04d8c23" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44489 } } peers { permanent_uuid: "7857dc6dacdd4c85ac07c64895391c52" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 35251 } } peers { permanent_uuid: "1100629d47e24c089b043f3f70597104" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37497 } }
I20260925 22:05:12.933027 5691 catalog_manager.cc:5495] T 5210bf68d0df44c9b70bc3b5308b12f7 P 7857dc6dacdd4c85ac07c64895391c52 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7857dc6dacdd4c85ac07c64895391c52 (127.5.0.65). New cstate: current_term: 1 leader_uuid: "7857dc6dacdd4c85ac07c64895391c52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5a0f52d78d4f9c9c1fe750f04d8c23" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7857dc6dacdd4c85ac07c64895391c52" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1100629d47e24c089b043f3f70597104" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37497 } health_report { overall_health: UNKNOWN } } }
W20260925 22:05:12.941883 6151 tablet.cc:2369] T 5210bf68d0df44c9b70bc3b5308b12f7 P 1100629d47e24c089b043f3f70597104: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 25 22:05:12 dist-test-slave-7mqr krb5kdc[5646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373911, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.65@KRBTEST.COM
I20260925 22:05:12.961615 6166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:12.953873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54990 (local address 127.5.0.65:35251)
0925 22:05:12.953952 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:12.953954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:12.954090 (+ 136us) server_negotiation.cc:407] Connection header received
0925 22:05:12.954196 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:12.954197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:12.954211 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:12.954295 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:12.955057 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.955580 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.956444 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.956657 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.956773 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:12.959011 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:12.959021 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:12.959022 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:12.959028 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:12.960657 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:12.960957 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:12.960960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:12.960960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:12.961020 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:12.961166 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:12.961166 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:12.961167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:12.961293 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:12.961398 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:12.961445 (+ 47us) server_negotiation.cc:299] Negotiation successful
0925 22:05:12.961504 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Sep 25 22:05:12 dist-test-slave-7mqr krb5kdc[5646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373912, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 22:05:12 dist-test-slave-7mqr krb5kdc[5646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373912, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.0.65@KRBTEST.COM
I20260925 22:05:12.983536 6166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:12.976923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55006 (local address 127.5.0.65:35251)
0925 22:05:12.976985 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:12.976988 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:12.977106 (+ 118us) server_negotiation.cc:407] Connection header received
0925 22:05:12.977153 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:12.977154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:12.977163 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:12.977210 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:12.977757 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.978224 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.978903 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:12.979076 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:12.979135 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:12.981259 (+ 2124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:12.981260 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:12.981260 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:12.981267 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:12.982612 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:12.982897 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:12.982898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:12.982898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:12.982945 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:12.983085 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:12.983085 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:12.983085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:12.983182 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:12.983266 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:12.983376 (+ 110us) server_negotiation.cc:299] Negotiation successful
0925 22:05:12.983435 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
Sep 25 22:05:12 dist-test-slave-7mqr krb5kdc[5646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 22:05:12 dist-test-slave-7mqr krb5kdc[5646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.65@KRBTEST.COM
I20260925 22:05:13.005537 6166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:12.998919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55022 (local address 127.5.0.65:35251)
0925 22:05:12.999005 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:12.999007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:12.999065 (+ 58us) server_negotiation.cc:407] Connection header received
0925 22:05:12.999123 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:12.999124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:12.999133 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:12.999182 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:12.999697 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:13.000226 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:13.000939 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:13.001144 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:13.001214 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:13.003350 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:13.003351 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:13.003351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:13.003358 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:13.004660 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:13.004955 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:13.004955 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:13.004956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:13.005004 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:13.005162 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:13.005162 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:13.005162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:13.005262 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:13.005356 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:13.005419 (+ 63us) server_negotiation.cc:299] Negotiation successful
0925 22:05:13.005472 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260925 22:05:13.005975 5796 tablet_service.cc:2908] Rejecting scan request for tablet 5210bf68d0df44c9b70bc3b5308b12f7: Uninitialized: clean time has not yet been initialized
I20260925 22:05:13.006842 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 5753
I20260925 22:05:13.011495 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 5887
I20260925 22:05:13.015635 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 6021
I20260925 22:05:13.019668 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 5659
2026-09-25T22:05:13Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2137 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:05:13 dist-test-slave-7mqr krb5kdc[6192](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:05:13 dist-test-slave-7mqr krb5kdc[6192](info): set up 2 sockets
Sep 25 22:05:13 dist-test-slave-7mqr krb5kdc[6192](info): commencing operation
krb5kdc: starting...
W20260925 22:05:14.063429 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:05:14 dist-test-slave-7mqr krb5kdc[6192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:05:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:05:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.0.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
I20260925 22:05:14.179159 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:33943
--webserver_interface=127.5.0.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:32823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:33943
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:14.260020 6205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:14.260229 6205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:14.260252 6205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:14.261430 6205 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:05:14.261472 6205 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 22:05:14.261485 6205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:14.261498 6205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:14.261511 6205 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:05:14.261530 6205 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:05:14.262969 6205 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:32823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:33943
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:33943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:14.263204 6205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:14.263465 6205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:14.268294 6211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:14.268294 6213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:14.268343 6210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:14.268667 6205 server_base.cc:1034] running on GCE node
I20260925 22:05:14.268896 6205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:14.269186 6205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:14.270354 6205 hybrid_clock.cc:648] HybridClock initialized: now 1790373914270392 us; error 98 us; skew 500 ppm
Sep 25 22:05:14 dist-test-slave-7mqr krb5kdc[6192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373914, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:14.278672 6205 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:05:14.279172 6205 webserver.cc:460] Webserver started at http://127.5.0.126:38361/ using document root <none> and password file <none>
I20260925 22:05:14.279404 6205 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:14.279472 6205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:14.279639 6205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:14.280808 6205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "64d1f9fa230447718ee1708b4c48b663"
format_stamp: "Formatted at 2026-09-25 22:05:14 on dist-test-slave-7mqr"
server_key: "2e12592d4d45699cab5019ec0c85434c"
server_key_iv: "398ff67628e634d883890ee46a285bb3"
server_key_version: "encryptionkey@0"
I20260925 22:05:14.281189 6205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "64d1f9fa230447718ee1708b4c48b663"
format_stamp: "Formatted at 2026-09-25 22:05:14 on dist-test-slave-7mqr"
server_key: "2e12592d4d45699cab5019ec0c85434c"
server_key_iv: "398ff67628e634d883890ee46a285bb3"
server_key_version: "encryptionkey@0"
I20260925 22:05:14.282567 6205 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 22:05:14.283335 6220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:14.283511 6205 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:05:14.283633 6205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "64d1f9fa230447718ee1708b4c48b663"
format_stamp: "Formatted at 2026-09-25 22:05:14 on dist-test-slave-7mqr"
server_key: "2e12592d4d45699cab5019ec0c85434c"
server_key_iv: "398ff67628e634d883890ee46a285bb3"
server_key_version: "encryptionkey@0"
I20260925 22:05:14.283721 6205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:14.297426 6205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:14.299261 6205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:14.299434 6205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:14.304260 6205 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:33943
I20260925 22:05:14.304315 6282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:33943 every 8 connection(s)
I20260925 22:05:14.304670 6205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:05:14.305317 6283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:14.307587 6283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64d1f9fa230447718ee1708b4c48b663: Bootstrap starting.
I20260925 22:05:14.308194 6283 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64d1f9fa230447718ee1708b4c48b663: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:14.308498 6283 log.cc:824] T 00000000000000000000000000000000 P 64d1f9fa230447718ee1708b4c48b663: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:14.309202 6283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64d1f9fa230447718ee1708b4c48b663: No bootstrap required, opened a new log
I20260925 22:05:14.311249 6283 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64d1f9fa230447718ee1708b4c48b663 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64d1f9fa230447718ee1708b4c48b663" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 33943 } }
I20260925 22:05:14.311388 6283 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64d1f9fa230447718ee1708b4c48b663 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:14.311410 6283 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64d1f9fa230447718ee1708b4c48b663 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64d1f9fa230447718ee1708b4c48b663, State: Initialized, Role: FOLLOWER
I20260925 22:05:14.311564 6283 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64d1f9fa230447718ee1708b4c48b663 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64d1f9fa230447718ee1708b4c48b663" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 33943 } }
I20260925 22:05:14.311643 6283 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64d1f9fa230447718ee1708b4c48b663 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:05:14.311686 6283 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64d1f9fa230447718ee1708b4c48b663 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:05:14.311738 6283 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64d1f9fa230447718ee1708b4c48b663 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:14.312345 6283 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64d1f9fa230447718ee1708b4c48b663 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64d1f9fa230447718ee1708b4c48b663" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 33943 } }
I20260925 22:05:14.312480 6283 leader_election.cc:302] T 00000000000000000000000000000000 P 64d1f9fa230447718ee1708b4c48b663 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64d1f9fa230447718ee1708b4c48b663; no voters:
I20260925 22:05:14.312655 6283 leader_election.cc:288] T 00000000000000000000000000000000 P 64d1f9fa230447718ee1708b4c48b663 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:05:14.312691 6286 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64d1f9fa230447718ee1708b4c48b663 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:14.312918 6283 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64d1f9fa230447718ee1708b4c48b663 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:05:14.312991 6286 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64d1f9fa230447718ee1708b4c48b663 [term 1 LEADER]: Becoming Leader. State: Replica: 64d1f9fa230447718ee1708b4c48b663, State: Running, Role: LEADER
I20260925 22:05:14.313167 6286 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64d1f9fa230447718ee1708b4c48b663 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64d1f9fa230447718ee1708b4c48b663" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 33943 } }
I20260925 22:05:14.313588 6288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64d1f9fa230447718ee1708b4c48b663 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64d1f9fa230447718ee1708b4c48b663. Latest consensus state: current_term: 1 leader_uuid: "64d1f9fa230447718ee1708b4c48b663" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64d1f9fa230447718ee1708b4c48b663" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 33943 } } }
I20260925 22:05:14.313683 6288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64d1f9fa230447718ee1708b4c48b663 [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:14.313606 6287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64d1f9fa230447718ee1708b4c48b663 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64d1f9fa230447718ee1708b4c48b663" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64d1f9fa230447718ee1708b4c48b663" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 33943 } } }
I20260925 22:05:14.313737 6287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64d1f9fa230447718ee1708b4c48b663 [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:14.313870 6295 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:05:14.314342 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 6205
I20260925 22:05:14.314440 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:05:14.314602 5121 external_mini_cluster.cc:1428] Setting key 04387307676f43b6817a33c626af6966
I20260925 22:05:14.314610 6295 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:05:14.316167 6295 catalog_manager.cc:1347] Generated new cluster ID: 4895bfb8b6cf4e3496773538277f53f2
I20260925 22:05:14.316226 6295 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:05:14 dist-test-slave-7mqr krb5kdc[6192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:14.326676 6305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:14.315167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55682 (local address 127.5.0.126:33943)
0925 22:05:14.315630 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:14.315640 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:14.315678 (+ 38us) server_negotiation.cc:407] Connection header received
0925 22:05:14.315797 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:14.315809 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:14.315867 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:14.316042 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:14.316894 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.318355 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.319371 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.320170 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.320294 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:14.322977 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:14.322981 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:14.322981 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:14.322993 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:14.324974 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:14.325393 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:14.325396 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:14.325396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:14.325511 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:14.325727 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:14.325728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:14.325728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:14.326011 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:14.326141 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:14.326279 (+ 138us) server_negotiation.cc:299] Negotiation successful
0925 22:05:14.326418 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":60,"threads_started":1}
I20260925 22:05:14.335700 6295 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:05:14.336290 6295 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:05:14.343690 6295 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64d1f9fa230447718ee1708b4c48b663: Generated new TSK 0
I20260925 22:05:14.343883 6295 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.0.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
I20260925 22:05:14.371011 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.65:0
--local_ip_for_outbound_sockets=127.5.0.65
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:33943
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:32823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:14.450632 6309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:14.450839 6309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:14.450866 6309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:14.452139 6309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:14.452184 6309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:14.452205 6309 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 22:05:14.452258 6309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.65
I20260925 22:05:14.453682 6309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:32823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.0.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.0.126:33943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.65
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:14.453933 6309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:14.454174 6309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:14.459088 6314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:14.459108 6317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:14.459100 6315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:14.459376 6309 server_base.cc:1034] running on GCE node
I20260925 22:05:14.459590 6309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:14.459841 6309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:14.461016 6309 hybrid_clock.cc:648] HybridClock initialized: now 1790373914461002 us; error 39 us; skew 500 ppm
Sep 25 22:05:14 dist-test-slave-7mqr krb5kdc[6192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373914, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:14.463522 6309 init.cc:377] Logged in from keytab as kudu/127.5.0.65@KRBTEST.COM (short username kudu)
I20260925 22:05:14.463958 6309 webserver.cc:460] Webserver started at http://127.5.0.65:43323/ using document root <none> and password file <none>
I20260925 22:05:14.464145 6309 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:14.464187 6309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:14.464285 6309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:14.465308 6309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-0/data/instance:
uuid: "9a83e105650f4178816ee98e36b6efad"
format_stamp: "Formatted at 2026-09-25 22:05:14 on dist-test-slave-7mqr"
server_key: "2355a1bb8025d31aab47383a273f27cb"
server_key_iv: "a9494656394dcb1e152e8e39730a116b"
server_key_version: "encryptionkey@0"
I20260925 22:05:14.465692 6309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance:
uuid: "9a83e105650f4178816ee98e36b6efad"
format_stamp: "Formatted at 2026-09-25 22:05:14 on dist-test-slave-7mqr"
server_key: "2355a1bb8025d31aab47383a273f27cb"
server_key_iv: "a9494656394dcb1e152e8e39730a116b"
server_key_version: "encryptionkey@0"
I20260925 22:05:14.466953 6309 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:05:14.467633 6324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:14.467800 6309 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:05:14.467921 6309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-0/wal
uuid: "9a83e105650f4178816ee98e36b6efad"
format_stamp: "Formatted at 2026-09-25 22:05:14 on dist-test-slave-7mqr"
server_key: "2355a1bb8025d31aab47383a273f27cb"
server_key_iv: "a9494656394dcb1e152e8e39730a116b"
server_key_version: "encryptionkey@0"
I20260925 22:05:14.467998 6309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:14.474912 6309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:14.476776 6309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:14.476883 6309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:14.477315 6309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:14.477355 6309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:14.477377 6309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:14.477423 6309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:14.483963 6309 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.65:46697
I20260925 22:05:14.484025 6439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.65:46697 every 8 connection(s)
I20260925 22:05:14.484380 6309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
Sep 25 22:05:14 dist-test-slave-7mqr krb5kdc[6192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373914, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:14.485845 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 6309
I20260925 22:05:14.485985 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance
I20260925 22:05:14.486181 5121 external_mini_cluster.cc:1428] Setting key 097f8b91aa0ff930816d12100d150de1
I20260925 22:05:14.488269 6305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:14.477832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:59737 (local address 127.5.0.126:33943)
0925 22:05:14.477911 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:14.477914 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:14.478070 (+ 156us) server_negotiation.cc:407] Connection header received
0925 22:05:14.478516 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:14.478518 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:14.478529 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:14.478621 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:14.479501 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.480354 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.482052 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.482295 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.482354 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:14.485090 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:14.485091 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:14.485092 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:14.485098 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:14.486712 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:14.487203 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:14.487205 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:14.487205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:14.487271 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:14.487516 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:14.487518 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:14.487518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:14.487664 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:14.487733 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:14.488021 (+ 288us) server_negotiation.cc:299] Negotiation successful
0925 22:05:14.488117 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 22:05:14.488299 6333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:14.477785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:33943 (local address 127.5.0.65:59737)
0925 22:05:14.477941 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:14.477980 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:14.478425 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:14.478702 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:14.478705 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:14.479086 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:14.479370 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:14.479378 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.480485 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.480486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:14.481954 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:14.481958 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.482390 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.482391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:14.482548 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:14.483166 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:14.483177 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:14.485003 (+ 1826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:14.486804 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:14.486811 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:14.486813 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:14.487082 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:14.487337 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:14.487337 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:14.487337 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:14.487431 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:14.487726 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:14.487730 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:14.487838 (+ 108us) client_negotiation.cc:769] Sending connection context
0925 22:05:14.487971 (+ 133us) client_negotiation.cc:240] Negotiation successful
0925 22:05:14.488110 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":92,"threads_started":1}
I20260925 22:05:14.489081 6440 heartbeater.cc:347] Connected to a master server at 127.5.0.126:33943
I20260925 22:05:14.489176 6440 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:14.489348 6440 heartbeater.cc:511] Master 127.5.0.126:33943 requested a full tablet report, sending...
I20260925 22:05:14.489814 6237 ts_manager.cc:194] Registered new tserver with Master: 9a83e105650f4178816ee98e36b6efad (127.5.0.65:46697)
I20260925 22:05:14.490674 6237 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.65@KRBTEST.COM'} at 127.5.0.65:59737
WARNING: no policy specified for kudu/127.5.0.66@KRBTEST.COM; defaulting to no policy
I20260925 22:05:14.496104 6447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:14.490115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:33943 (local address 127.5.0.65:46979)
0925 22:05:14.490284 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:14.490295 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:14.490353 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:14.490470 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:14.490470 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:14.490665 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:14.490732 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:14.490734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.491621 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.491622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:14.492175 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:14.492176 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.492684 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.492685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:14.492754 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:14.493190 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:14.493195 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:14.493468 (+ 273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:14.495159 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:14.495160 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:14.495160 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:14.495315 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:14.495510 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:14.495511 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:14.495511 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:14.495558 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:14.495834 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:14.495835 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:14.495866 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 22:05:14.495958 (+ 92us) client_negotiation.cc:240] Negotiation successful
0925 22:05:14.496034 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":119,"threads_started":1}
I20260925 22:05:14.496109 6305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:14.490175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:46979 (local address 127.5.0.126:33943)
0925 22:05:14.490214 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:14.490215 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:14.490361 (+ 146us) server_negotiation.cc:407] Connection header received
0925 22:05:14.490392 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:14.490393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:14.490403 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:14.490465 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:14.490823 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.491513 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.492249 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.492588 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.492674 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:14.493557 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:14.493558 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:14.493559 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:14.493566 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:14.495080 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:14.495383 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:14.495384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:14.495385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:14.495442 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:14.495639 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:14.495640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:14.495640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:14.495770 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:14.495845 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:14.495996 (+ 151us) server_negotiation.cc:299] Negotiation successful
0925 22:05:14.496055 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
I20260925 22:05:14.512938 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.66:0
--local_ip_for_outbound_sockets=127.5.0.66
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:33943
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:32823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:14.595291 6449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:14.595498 6449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:14.595526 6449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:14.596616 6449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:14.596649 6449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:14.596669 6449 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 22:05:14.596688 6449 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.66
I20260925 22:05:14.598067 6449 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:32823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.0.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.0.126:33943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.66
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:14.598249 6449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:14.598407 6449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:14.603138 6457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:14.603168 6449 server_base.cc:1034] running on GCE node
W20260925 22:05:14.603142 6455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:14.603183 6454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:14.603571 6449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:14.603814 6449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:14.604969 6449 hybrid_clock.cc:648] HybridClock initialized: now 1790373914604967 us; error 56 us; skew 500 ppm
Sep 25 22:05:14 dist-test-slave-7mqr krb5kdc[6192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373914, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:14.607259 6449 init.cc:377] Logged in from keytab as kudu/127.5.0.66@KRBTEST.COM (short username kudu)
I20260925 22:05:14.607726 6449 webserver.cc:460] Webserver started at http://127.5.0.66:40511/ using document root <none> and password file <none>
I20260925 22:05:14.608006 6449 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:14.608071 6449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:14.608206 6449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:14.609205 6449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-1/data/instance:
uuid: "febea1ee6a3446f39797cf6c8b74d995"
format_stamp: "Formatted at 2026-09-25 22:05:14 on dist-test-slave-7mqr"
server_key: "f1f0f1ae6fffa1cc6783279f54f9a4f9"
server_key_iv: "3a3a23d8120a690eb7dac3cb66757337"
server_key_version: "encryptionkey@0"
I20260925 22:05:14.609688 6449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance:
uuid: "febea1ee6a3446f39797cf6c8b74d995"
format_stamp: "Formatted at 2026-09-25 22:05:14 on dist-test-slave-7mqr"
server_key: "f1f0f1ae6fffa1cc6783279f54f9a4f9"
server_key_iv: "3a3a23d8120a690eb7dac3cb66757337"
server_key_version: "encryptionkey@0"
I20260925 22:05:14.611060 6449 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:05:14.611822 6464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:14.612041 6449 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:05:14.612142 6449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-1/wal
uuid: "febea1ee6a3446f39797cf6c8b74d995"
format_stamp: "Formatted at 2026-09-25 22:05:14 on dist-test-slave-7mqr"
server_key: "f1f0f1ae6fffa1cc6783279f54f9a4f9"
server_key_iv: "3a3a23d8120a690eb7dac3cb66757337"
server_key_version: "encryptionkey@0"
I20260925 22:05:14.612222 6449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:14.632723 6449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:14.634651 6449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:14.634814 6449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:14.635326 6449 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:14.635391 6449 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:14.635456 6449 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:14.635494 6449 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:14.642715 6449 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.66:33975
I20260925 22:05:14.642839 6579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.66:33975 every 8 connection(s)
Sep 25 22:05:14 dist-test-slave-7mqr krb5kdc[6192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373914, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:14.643131 6449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
I20260925 22:05:14.646216 6305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:14.635926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:56345 (local address 127.5.0.126:33943)
0925 22:05:14.635977 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:14.635979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:14.636169 (+ 190us) server_negotiation.cc:407] Connection header received
0925 22:05:14.636634 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:14.636635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:14.636648 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:14.636730 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:14.637369 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.638158 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.639848 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.640161 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.640260 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:14.643589 (+ 3329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:14.643591 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:14.643592 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:14.643600 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:14.644842 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:14.645252 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:14.645253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:14.645254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:14.645318 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:14.645574 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:14.645575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:14.645575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:14.645702 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:14.645780 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:14.646047 (+ 267us) server_negotiation.cc:299] Negotiation successful
0925 22:05:14.646126 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 22:05:14.646391 6473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:14.635860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:33943 (local address 127.5.0.66:56345)
0925 22:05:14.636026 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:14.636065 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:14.636544 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:14.636794 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:14.636797 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:14.637058 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:14.637275 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:14.637279 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.638259 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.638260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:14.639736 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:14.639740 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.640333 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.640334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:14.640470 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:14.641611 (+ 1141us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:14.641620 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:14.643505 (+ 1885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:14.644928 (+ 1423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:14.644934 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:14.644936 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:14.645162 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:14.645384 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:14.645385 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:14.645385 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:14.645504 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:14.645770 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:14.645775 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:14.645913 (+ 138us) client_negotiation.cc:769] Sending connection context
0925 22:05:14.646013 (+ 100us) client_negotiation.cc:240] Negotiation successful
0925 22:05:14.646173 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":102,"threads_started":1}
I20260925 22:05:14.646886 6580 heartbeater.cc:347] Connected to a master server at 127.5.0.126:33943
I20260925 22:05:14.646971 6580 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:14.647148 6580 heartbeater.cc:511] Master 127.5.0.126:33943 requested a full tablet report, sending...
I20260925 22:05:14.647579 6236 ts_manager.cc:194] Registered new tserver with Master: febea1ee6a3446f39797cf6c8b74d995 (127.5.0.66:33975)
I20260925 22:05:14.648130 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 6449
I20260925 22:05:14.648168 6236 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.66@KRBTEST.COM'} at 127.5.0.66:56345
I20260925 22:05:14.648219 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance
I20260925 22:05:14.648414 5121 external_mini_cluster.cc:1428] Setting key dbdadb8445d58be64da90db57ed38ed3
I20260925 22:05:14.654071 6305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:14.647936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:60557 (local address 127.5.0.126:33943)
0925 22:05:14.648346 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:14.648349 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:14.648357 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:05:14.648375 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:14.648376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:14.648387 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:14.648439 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:14.648780 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.649382 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.650422 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.650607 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.650658 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:14.651629 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:14.651631 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:14.651631 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:14.651637 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:14.652992 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:14.653376 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:14.653377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:14.653377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:14.653423 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:14.653623 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:14.653624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:14.653624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:14.653745 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:14.653794 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:14.653963 (+ 169us) server_negotiation.cc:299] Negotiation successful
0925 22:05:14.654015 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390}
I20260925 22:05:14.654227 6586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:14.647856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:33943 (local address 127.5.0.66:60557)
0925 22:05:14.647991 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:14.648001 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:14.648055 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:14.648454 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:14.648455 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:14.648657 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:14.648712 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:14.648713 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.649476 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.649477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:14.650337 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:14.650340 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.650679 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.650680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:14.650765 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:14.651146 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:14.651151 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:14.651544 (+ 393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:14.653068 (+ 1524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:14.653069 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:14.653069 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:14.653301 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:14.653484 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:14.653485 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:14.653486 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:14.653555 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:14.653810 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:14.653811 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:14.653861 (+ 50us) client_negotiation.cc:769] Sending connection context
0925 22:05:14.653948 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 22:05:14.654064 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":87,"threads_started":1}
WARNING: no policy specified for kudu/127.5.0.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
I20260925 22:05:14.673022 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.67:0
--local_ip_for_outbound_sockets=127.5.0.67
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:33943
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:32823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:14.753849 6589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:14.754091 6589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:14.754130 6589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:14.755292 6589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:14.755362 6589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:14.755396 6589 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 22:05:14.755434 6589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.67
I20260925 22:05:14.756886 6589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:32823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.0.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.0.126:33943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.67
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:14.757156 6589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:14.757385 6589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:14.762315 6597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:14.762336 6594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:14.762319 6595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:14.762537 6589 server_base.cc:1034] running on GCE node
I20260925 22:05:14.762722 6589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:14.763005 6589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:14.764185 6589 hybrid_clock.cc:648] HybridClock initialized: now 1790373914764166 us; error 46 us; skew 500 ppm
Sep 25 22:05:14 dist-test-slave-7mqr krb5kdc[6192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373914, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:14.766678 6589 init.cc:377] Logged in from keytab as kudu/127.5.0.67@KRBTEST.COM (short username kudu)
I20260925 22:05:14.767179 6589 webserver.cc:460] Webserver started at http://127.5.0.67:38399/ using document root <none> and password file <none>
I20260925 22:05:14.767417 6589 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:14.767485 6589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:14.767644 6589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:14.768765 6589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-2/data/instance:
uuid: "6457322f1e6c41ed82e7bb0068728d8e"
format_stamp: "Formatted at 2026-09-25 22:05:14 on dist-test-slave-7mqr"
server_key: "ad13a567aa416c09b53096d4a570d935"
server_key_iv: "92959895edfd099aabdc23ad40a21ea5"
server_key_version: "encryptionkey@0"
I20260925 22:05:14.769137 6589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance:
uuid: "6457322f1e6c41ed82e7bb0068728d8e"
format_stamp: "Formatted at 2026-09-25 22:05:14 on dist-test-slave-7mqr"
server_key: "ad13a567aa416c09b53096d4a570d935"
server_key_iv: "92959895edfd099aabdc23ad40a21ea5"
server_key_version: "encryptionkey@0"
I20260925 22:05:14.770429 6589 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 22:05:14.771205 6604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:14.771376 6589 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:05:14.771479 6589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-2/wal
uuid: "6457322f1e6c41ed82e7bb0068728d8e"
format_stamp: "Formatted at 2026-09-25 22:05:14 on dist-test-slave-7mqr"
server_key: "ad13a567aa416c09b53096d4a570d935"
server_key_iv: "92959895edfd099aabdc23ad40a21ea5"
server_key_version: "encryptionkey@0"
I20260925 22:05:14.771555 6589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:14.788578 6589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:14.790623 6589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:14.790789 6589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:14.791430 6589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:14.791487 6589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:14.791522 6589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:14.791549 6589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 22:05:14 dist-test-slave-7mqr krb5kdc[6192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373914, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:14.799104 6589 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.67:44357
I20260925 22:05:14.799216 6719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.67:44357 every 8 connection(s)
I20260925 22:05:14.799511 6589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
I20260925 22:05:14.801091 6305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:14.791816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:59011 (local address 127.5.0.126:33943)
0925 22:05:14.791871 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:14.791875 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:14.792135 (+ 260us) server_negotiation.cc:407] Connection header received
0925 22:05:14.792867 (+ 732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:14.792868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:14.792879 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:14.792922 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:14.793846 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.794391 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.795416 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.795624 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.795710 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:14.798047 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:14.798048 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:14.798049 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:14.798057 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:14.799802 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:14.800281 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:14.800282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:14.800282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:14.800337 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:14.800609 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:14.800610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:14.800610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:14.800712 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:14.800763 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:14.800968 (+ 205us) server_negotiation.cc:299] Negotiation successful
0925 22:05:14.801029 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 22:05:14.801231 6613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:14.791814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:33943 (local address 127.5.0.67:59011)
0925 22:05:14.791984 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:14.792029 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:14.792762 (+ 733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:14.793001 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:14.793007 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:14.793418 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:14.793720 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:14.793728 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.794489 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.794490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:14.795316 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:14.795318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.795721 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.795722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:14.795880 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:14.796495 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:14.796503 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:14.797948 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:14.799929 (+ 1981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:14.799934 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:14.799936 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:14.800214 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:14.800415 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:14.800415 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:14.800416 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:14.800542 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:14.800754 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:14.800756 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:14.800861 (+ 105us) client_negotiation.cc:769] Sending connection context
0925 22:05:14.800929 (+ 68us) client_negotiation.cc:240] Negotiation successful
0925 22:05:14.801041 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":101,"threads_started":1}
I20260925 22:05:14.801921 6720 heartbeater.cc:347] Connected to a master server at 127.5.0.126:33943
I20260925 22:05:14.801990 6720 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:14.802146 6720 heartbeater.cc:511] Master 127.5.0.126:33943 requested a full tablet report, sending...
I20260925 22:05:14.802506 6237 ts_manager.cc:194] Registered new tserver with Master: 6457322f1e6c41ed82e7bb0068728d8e (127.5.0.67:44357)
I20260925 22:05:14.802981 6237 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.67@KRBTEST.COM'} at 127.5.0.67:59011
I20260925 22:05:14.807523 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 6589
I20260925 22:05:14.807632 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance
I20260925 22:05:14.807808 5121 external_mini_cluster.cc:1428] Setting key 87398f4d806b46239f1abcfe8f5af31f
I20260925 22:05:14.807883 6305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:14.803003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:34329 (local address 127.5.0.126:33943)
0925 22:05:14.803066 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:14.803068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:14.803213 (+ 145us) server_negotiation.cc:407] Connection header received
0925 22:05:14.803267 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:14.803268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:14.803278 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:14.803338 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:14.803689 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.804197 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.804830 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.804997 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.805043 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:14.805783 (+ 740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:14.805785 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:14.805785 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:14.805790 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:14.806860 (+ 1070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:14.807148 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:14.807149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:14.807149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:14.807194 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:14.807322 (+ 128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:14.807323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:14.807323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:14.807467 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:14.807541 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:14.807716 (+ 175us) server_negotiation.cc:299] Negotiation successful
0925 22:05:14.807792 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 22:05:14.808202 6726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:14.802972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:33943 (local address 127.5.0.67:34329)
0925 22:05:14.803141 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:14.803152 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:14.803206 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:14.803372 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:14.803373 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:14.803562 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:14.803615 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:14.803617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.804279 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.804280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:14.804765 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:14.804766 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.805056 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.805057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:14.805111 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:14.805440 (+ 329us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:14.805444 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:14.805696 (+ 252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:14.806924 (+ 1228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:14.806925 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:14.806925 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:14.807084 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:14.807234 (+ 150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:14.807234 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:14.807235 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:14.807275 (+ 40us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:14.807537 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:14.807537 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:14.807570 (+ 33us) client_negotiation.cc:769] Sending connection context
0925 22:05:14.807984 (+ 414us) client_negotiation.cc:240] Negotiation successful
0925 22:05:14.808089 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":118,"threads_started":1}
I20260925 22:05:14.808733 5121 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 22:05:14 dist-test-slave-7mqr krb5kdc[6192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373914, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 22:05:14 dist-test-slave-7mqr krb5kdc[6192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373914, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:14.840692 6305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:14.832335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55692 (local address 127.5.0.126:33943)
0925 22:05:14.832386 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:14.832388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:14.832398 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:05:14.832431 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:14.832432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:14.832444 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:14.832520 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:14.833239 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.834028 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.835784 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.836025 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.836079 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:14.838450 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:14.838451 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:14.838452 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:14.838458 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:14.839600 (+ 1142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:14.840050 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:14.840051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:14.840051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:14.840105 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:14.840274 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:14.840275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:14.840275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:14.840379 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:14.840448 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:14.840542 (+ 94us) server_negotiation.cc:299] Negotiation successful
0925 22:05:14.840600 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 22:05:14.842326 6237 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55692:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:05:14.844022 6237 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:05:14.850395 6740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:14.846576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:44357 (local address 127.0.0.1:41176)
0925 22:05:14.846802 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:14.846812 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:14.846865 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:14.847329 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:14.847330 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:14.847332 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:14.847385 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:14.847386 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.848322 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.848323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:14.849337 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:14.849339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.850091 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.850091 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:14.850166 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:14.850181 (+ 15us) client_negotiation.cc:769] Sending connection context
0925 22:05:14.850265 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 22:05:14.850317 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":41,"thread_start_us":67,"threads_started":1}
I20260925 22:05:14.850420 6742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:14.846812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41176 (local address 127.5.0.67:44357)
0925 22:05:14.847126 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:14.847130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:14.847139 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:05:14.847175 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:14.847177 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:14.847193 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:14.847264 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:14.847447 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.848241 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.849431 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.850003 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.850079 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:14.850144 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:14.850286 (+ 142us) server_negotiation.cc:299] Negotiation successful
0925 22:05:14.850322 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":90,"threads_started":1}
I20260925 22:05:14.850900 6738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:14.846366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:46697 (local address 127.0.0.1:45538)
0925 22:05:14.846555 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:14.846599 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:14.846665 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:14.846994 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:14.846997 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:14.846999 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:14.847078 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:14.847081 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.848482 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.848482 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:14.849739 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:14.849742 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.850655 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.850656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:14.850736 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:14.850740 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:05:14.850793 (+ 53us) client_negotiation.cc:240] Negotiation successful
0925 22:05:14.850835 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":67,"thread_start_us":53,"threads_started":1}
I20260925 22:05:14.850955 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:14.846464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45538 (local address 127.5.0.65:46697)
0925 22:05:14.846837 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:14.846842 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:14.846852 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:05:14.846894 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:14.846896 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:14.846911 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:14.846987 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:14.847177 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.848386 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.849827 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.850573 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.850651 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:14.850705 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:14.850815 (+ 110us) server_negotiation.cc:299] Negotiation successful
0925 22:05:14.850854 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":88,"threads_started":1}
I20260925 22:05:14.850965 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:14.846556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40540 (local address 127.5.0.66:33975)
0925 22:05:14.846722 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:14.846757 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:14.846872 (+ 115us) server_negotiation.cc:407] Connection header received
0925 22:05:14.846910 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:14.846913 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:14.846927 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:14.846999 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:14.847177 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.848362 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.849839 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.850507 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.850547 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:14.850598 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:14.850829 (+ 231us) server_negotiation.cc:299] Negotiation successful
0925 22:05:14.850867 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":90,"threads_started":1}
I20260925 22:05:14.851123 6737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:14.846377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:33975 (local address 127.0.0.1:40540)
0925 22:05:14.846756 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:14.846766 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:14.846816 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:14.846996 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:14.846998 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:14.846999 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:14.847078 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:14.847080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.848482 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.848483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:14.849741 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:14.849743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.850675 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.850676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:14.850736 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:14.850739 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:05:14.851036 (+ 297us) client_negotiation.cc:240] Negotiation successful
0925 22:05:14.851073 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":66,"thread_start_us":89,"threads_started":1}
I20260925 22:05:14.851488 6654 tablet_service.cc:1467] Processing CreateTablet for tablet f1ca445061924327b6ed5a886584eea3 (DEFAULT_TABLE table=test-table [id=0905beacf8cb40f6a6d5c9514c302f0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:14.851575 6374 tablet_service.cc:1467] Processing CreateTablet for tablet f1ca445061924327b6ed5a886584eea3 (DEFAULT_TABLE table=test-table [id=0905beacf8cb40f6a6d5c9514c302f0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:14.851857 6654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1ca445061924327b6ed5a886584eea3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:14.851881 6374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1ca445061924327b6ed5a886584eea3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:14.852531 6514 tablet_service.cc:1467] Processing CreateTablet for tablet f1ca445061924327b6ed5a886584eea3 (DEFAULT_TABLE table=test-table [id=0905beacf8cb40f6a6d5c9514c302f0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:14.852855 6514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1ca445061924327b6ed5a886584eea3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:14.854823 6743 tablet_bootstrap.cc:492] T f1ca445061924327b6ed5a886584eea3 P 6457322f1e6c41ed82e7bb0068728d8e: Bootstrap starting.
I20260925 22:05:14.855762 6743 tablet_bootstrap.cc:654] T f1ca445061924327b6ed5a886584eea3 P 6457322f1e6c41ed82e7bb0068728d8e: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:14.855794 6744 tablet_bootstrap.cc:492] T f1ca445061924327b6ed5a886584eea3 P 9a83e105650f4178816ee98e36b6efad: Bootstrap starting.
I20260925 22:05:14.856280 6743 log.cc:824] T f1ca445061924327b6ed5a886584eea3 P 6457322f1e6c41ed82e7bb0068728d8e: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:14.856328 6745 tablet_bootstrap.cc:492] T f1ca445061924327b6ed5a886584eea3 P febea1ee6a3446f39797cf6c8b74d995: Bootstrap starting.
I20260925 22:05:14.856642 6744 tablet_bootstrap.cc:654] T f1ca445061924327b6ed5a886584eea3 P 9a83e105650f4178816ee98e36b6efad: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:14.856945 6745 tablet_bootstrap.cc:654] T f1ca445061924327b6ed5a886584eea3 P febea1ee6a3446f39797cf6c8b74d995: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:14.857002 6744 log.cc:824] T f1ca445061924327b6ed5a886584eea3 P 9a83e105650f4178816ee98e36b6efad: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:14.857153 6743 tablet_bootstrap.cc:492] T f1ca445061924327b6ed5a886584eea3 P 6457322f1e6c41ed82e7bb0068728d8e: No bootstrap required, opened a new log
I20260925 22:05:14.857224 6743 ts_tablet_manager.cc:1397] T f1ca445061924327b6ed5a886584eea3 P 6457322f1e6c41ed82e7bb0068728d8e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:05:14.857297 6745 log.cc:824] T f1ca445061924327b6ed5a886584eea3 P febea1ee6a3446f39797cf6c8b74d995: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:14.858047 6745 tablet_bootstrap.cc:492] T f1ca445061924327b6ed5a886584eea3 P febea1ee6a3446f39797cf6c8b74d995: No bootstrap required, opened a new log
I20260925 22:05:14.858100 6745 ts_tablet_manager.cc:1397] T f1ca445061924327b6ed5a886584eea3 P febea1ee6a3446f39797cf6c8b74d995: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:05:14.859128 6743 raft_consensus.cc:348] T f1ca445061924327b6ed5a886584eea3 P 6457322f1e6c41ed82e7bb0068728d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } }
I20260925 22:05:14.859308 6743 raft_consensus.cc:374] T f1ca445061924327b6ed5a886584eea3 P 6457322f1e6c41ed82e7bb0068728d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:14.859354 6743 raft_consensus.cc:729] T f1ca445061924327b6ed5a886584eea3 P 6457322f1e6c41ed82e7bb0068728d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6457322f1e6c41ed82e7bb0068728d8e, State: Initialized, Role: FOLLOWER
I20260925 22:05:14.859469 6743 consensus_queue.cc:260] T f1ca445061924327b6ed5a886584eea3 P 6457322f1e6c41ed82e7bb0068728d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } }
I20260925 22:05:14.859781 6743 ts_tablet_manager.cc:1428] T f1ca445061924327b6ed5a886584eea3 P 6457322f1e6c41ed82e7bb0068728d8e: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260925 22:05:14.859807 6720 heartbeater.cc:503] Master 127.5.0.126:33943 was elected leader, sending a full tablet report...
I20260925 22:05:14.859797 6745 raft_consensus.cc:348] T f1ca445061924327b6ed5a886584eea3 P febea1ee6a3446f39797cf6c8b74d995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } }
I20260925 22:05:14.859997 6745 raft_consensus.cc:374] T f1ca445061924327b6ed5a886584eea3 P febea1ee6a3446f39797cf6c8b74d995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:14.860040 6745 raft_consensus.cc:729] T f1ca445061924327b6ed5a886584eea3 P febea1ee6a3446f39797cf6c8b74d995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: febea1ee6a3446f39797cf6c8b74d995, State: Initialized, Role: FOLLOWER
I20260925 22:05:14.860198 6745 consensus_queue.cc:260] T f1ca445061924327b6ed5a886584eea3 P febea1ee6a3446f39797cf6c8b74d995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } }
I20260925 22:05:14.860405 6744 tablet_bootstrap.cc:492] T f1ca445061924327b6ed5a886584eea3 P 9a83e105650f4178816ee98e36b6efad: No bootstrap required, opened a new log
I20260925 22:05:14.860462 6744 ts_tablet_manager.cc:1397] T f1ca445061924327b6ed5a886584eea3 P 9a83e105650f4178816ee98e36b6efad: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260925 22:05:14.860463 6745 ts_tablet_manager.cc:1428] T f1ca445061924327b6ed5a886584eea3 P febea1ee6a3446f39797cf6c8b74d995: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 22:05:14.860474 6580 heartbeater.cc:503] Master 127.5.0.126:33943 was elected leader, sending a full tablet report...
I20260925 22:05:14.862507 6744 raft_consensus.cc:348] T f1ca445061924327b6ed5a886584eea3 P 9a83e105650f4178816ee98e36b6efad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } }
I20260925 22:05:14.862614 6744 raft_consensus.cc:374] T f1ca445061924327b6ed5a886584eea3 P 9a83e105650f4178816ee98e36b6efad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:14.862634 6744 raft_consensus.cc:729] T f1ca445061924327b6ed5a886584eea3 P 9a83e105650f4178816ee98e36b6efad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a83e105650f4178816ee98e36b6efad, State: Initialized, Role: FOLLOWER
I20260925 22:05:14.862706 6744 consensus_queue.cc:260] T f1ca445061924327b6ed5a886584eea3 P 9a83e105650f4178816ee98e36b6efad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } }
I20260925 22:05:14.862883 6744 ts_tablet_manager.cc:1428] T f1ca445061924327b6ed5a886584eea3 P 9a83e105650f4178816ee98e36b6efad: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 22:05:14.862949 6440 heartbeater.cc:503] Master 127.5.0.126:33943 was elected leader, sending a full tablet report...
W20260925 22:05:14.893707 6581 tablet.cc:2369] T f1ca445061924327b6ed5a886584eea3 P febea1ee6a3446f39797cf6c8b74d995: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:05:14.933290 6750 raft_consensus.cc:482] T f1ca445061924327b6ed5a886584eea3 P febea1ee6a3446f39797cf6c8b74d995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:05:14.933465 6750 raft_consensus.cc:504] T f1ca445061924327b6ed5a886584eea3 P febea1ee6a3446f39797cf6c8b74d995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } }
I20260925 22:05:14.933854 6750 leader_election.cc:288] T f1ca445061924327b6ed5a886584eea3 P febea1ee6a3446f39797cf6c8b74d995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a83e105650f4178816ee98e36b6efad (127.5.0.65:46697), 6457322f1e6c41ed82e7bb0068728d8e (127.5.0.67:44357)
I20260925 22:05:14.937891 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:14.934235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:42317 (local address 127.5.0.65:46697)
0925 22:05:14.934287 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:14.934289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:14.934301 (+ 12us) server_negotiation.cc:407] Connection header received
0925 22:05:14.934372 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:14.934373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:14.934384 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:14.934446 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:14.934643 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.935416 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.936886 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.937468 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.937580 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:14.937590 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:14.937777 (+ 187us) server_negotiation.cc:299] Negotiation successful
0925 22:05:14.937825 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260925 22:05:14.937891 6742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:14.934360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:50563 (local address 127.5.0.67:44357)
0925 22:05:14.934411 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:14.934413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:14.934534 (+ 121us) server_negotiation.cc:407] Connection header received
0925 22:05:14.934576 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:14.934577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:14.934586 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:14.934661 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:14.934805 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.935500 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.936885 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.937469 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.937559 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:14.937572 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:14.937777 (+ 205us) server_negotiation.cc:299] Negotiation successful
0925 22:05:14.937824 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 22:05:14.937896 6752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:14.934258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:44357 (local address 127.5.0.66:50563)
0925 22:05:14.934447 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:14.934457 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:14.934522 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:14.934679 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:14.934679 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:14.934681 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:14.934740 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:14.934741 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.935570 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.935571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:14.936769 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:14.936770 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.937586 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.937586 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:14.937671 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:14.937675 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:05:14.937755 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 22:05:14.937797 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":6,"thread_start_us":108,"threads_started":1}
I20260925 22:05:14.938027 6473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:14.934122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:46697 (local address 127.5.0.66:42317)
0925 22:05:14.934214 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:14.934229 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:14.934299 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:14.934475 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:14.934476 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:14.934478 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:14.934557 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:14.934559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.935495 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.935496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:14.936768 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:14.936770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.937585 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.937586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:14.937677 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:14.937679 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 22:05:14.937740 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 22:05:14.937798 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"mutex_wait_us":21}
I20260925 22:05:14.938388 6674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1ca445061924327b6ed5a886584eea3" candidate_uuid: "febea1ee6a3446f39797cf6c8b74d995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6457322f1e6c41ed82e7bb0068728d8e" is_pre_election: true
I20260925 22:05:14.938436 6394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1ca445061924327b6ed5a886584eea3" candidate_uuid: "febea1ee6a3446f39797cf6c8b74d995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a83e105650f4178816ee98e36b6efad" is_pre_election: true
I20260925 22:05:14.938555 6674 raft_consensus.cc:2454] T f1ca445061924327b6ed5a886584eea3 P 6457322f1e6c41ed82e7bb0068728d8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate febea1ee6a3446f39797cf6c8b74d995 in term 0.
I20260925 22:05:14.938582 6394 raft_consensus.cc:2454] T f1ca445061924327b6ed5a886584eea3 P 9a83e105650f4178816ee98e36b6efad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate febea1ee6a3446f39797cf6c8b74d995 in term 0.
I20260925 22:05:14.938796 6467 leader_election.cc:302] T f1ca445061924327b6ed5a886584eea3 P febea1ee6a3446f39797cf6c8b74d995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a83e105650f4178816ee98e36b6efad, febea1ee6a3446f39797cf6c8b74d995; no voters:
I20260925 22:05:14.938925 6750 raft_consensus.cc:2781] T f1ca445061924327b6ed5a886584eea3 P febea1ee6a3446f39797cf6c8b74d995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:05:14.939013 6750 raft_consensus.cc:482] T f1ca445061924327b6ed5a886584eea3 P febea1ee6a3446f39797cf6c8b74d995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:05:14.939047 6750 raft_consensus.cc:3037] T f1ca445061924327b6ed5a886584eea3 P febea1ee6a3446f39797cf6c8b74d995 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:14.940081 6750 raft_consensus.cc:504] T f1ca445061924327b6ed5a886584eea3 P febea1ee6a3446f39797cf6c8b74d995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } }
I20260925 22:05:14.940220 6750 leader_election.cc:288] T f1ca445061924327b6ed5a886584eea3 P febea1ee6a3446f39797cf6c8b74d995 [CANDIDATE]: Term 1 election: Requested vote from peers 9a83e105650f4178816ee98e36b6efad (127.5.0.65:46697), 6457322f1e6c41ed82e7bb0068728d8e (127.5.0.67:44357)
I20260925 22:05:14.940416 6394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1ca445061924327b6ed5a886584eea3" candidate_uuid: "febea1ee6a3446f39797cf6c8b74d995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a83e105650f4178816ee98e36b6efad"
I20260925 22:05:14.940407 6674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1ca445061924327b6ed5a886584eea3" candidate_uuid: "febea1ee6a3446f39797cf6c8b74d995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6457322f1e6c41ed82e7bb0068728d8e"
I20260925 22:05:14.940485 6394 raft_consensus.cc:3037] T f1ca445061924327b6ed5a886584eea3 P 9a83e105650f4178816ee98e36b6efad [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:14.940503 6674 raft_consensus.cc:3037] T f1ca445061924327b6ed5a886584eea3 P 6457322f1e6c41ed82e7bb0068728d8e [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:14.941167 6394 raft_consensus.cc:2454] T f1ca445061924327b6ed5a886584eea3 P 9a83e105650f4178816ee98e36b6efad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate febea1ee6a3446f39797cf6c8b74d995 in term 1.
I20260925 22:05:14.941171 6674 raft_consensus.cc:2454] T f1ca445061924327b6ed5a886584eea3 P 6457322f1e6c41ed82e7bb0068728d8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate febea1ee6a3446f39797cf6c8b74d995 in term 1.
I20260925 22:05:14.941349 6467 leader_election.cc:302] T f1ca445061924327b6ed5a886584eea3 P febea1ee6a3446f39797cf6c8b74d995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a83e105650f4178816ee98e36b6efad, febea1ee6a3446f39797cf6c8b74d995; no voters:
I20260925 22:05:14.941452 6750 raft_consensus.cc:2781] T f1ca445061924327b6ed5a886584eea3 P febea1ee6a3446f39797cf6c8b74d995 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:14.941596 6750 raft_consensus.cc:686] T f1ca445061924327b6ed5a886584eea3 P febea1ee6a3446f39797cf6c8b74d995 [term 1 LEADER]: Becoming Leader. State: Replica: febea1ee6a3446f39797cf6c8b74d995, State: Running, Role: LEADER
I20260925 22:05:14.941695 6750 consensus_queue.cc:237] T f1ca445061924327b6ed5a886584eea3 P febea1ee6a3446f39797cf6c8b74d995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } }
I20260925 22:05:14.942577 6236 catalog_manager.cc:5495] T f1ca445061924327b6ed5a886584eea3 P febea1ee6a3446f39797cf6c8b74d995 reported cstate change: term changed from 0 to 1, leader changed from <none> to febea1ee6a3446f39797cf6c8b74d995 (127.5.0.66). New cstate: current_term: 1 leader_uuid: "febea1ee6a3446f39797cf6c8b74d995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } health_report { overall_health: UNKNOWN } } }
I20260925 22:05:14.966099 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:14.963953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40546 (local address 127.5.0.66:33975)
0925 22:05:14.963994 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:14.963996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:14.964004 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:05:14.964056 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:14.964057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:14.964073 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:14.964133 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:05:14.964248 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.964701 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.965406 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:14.965588 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:14.965642 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:14.965821 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:05:14.965910 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:05:14.965967 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:14.966004 (+ 37us) server_negotiation.cc:299] Negotiation successful
0925 22:05:14.966021 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:05:14.967803 6394 raft_consensus.cc:1261] T f1ca445061924327b6ed5a886584eea3 P 9a83e105650f4178816ee98e36b6efad [term 1 FOLLOWER]: Refusing update from remote peer febea1ee6a3446f39797cf6c8b74d995: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:05:14.967923 6674 raft_consensus.cc:1261] T f1ca445061924327b6ed5a886584eea3 P 6457322f1e6c41ed82e7bb0068728d8e [term 1 FOLLOWER]: Refusing update from remote peer febea1ee6a3446f39797cf6c8b74d995: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:05:14.968142 6753 consensus_queue.cc:1035] T f1ca445061924327b6ed5a886584eea3 P febea1ee6a3446f39797cf6c8b74d995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:05:14.968298 6750 consensus_queue.cc:1035] T f1ca445061924327b6ed5a886584eea3 P febea1ee6a3446f39797cf6c8b74d995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:05:14.970568 6757 mvcc.cc:205] Tried to move back new op lower bound from 7333371555705544704 to 7333371555601747968. Current Snapshot: MvccSnapshot[applied={T|T < 7333371555705544704}]
I20260925 22:05:14.970558 6756 mvcc.cc:205] Tried to move back new op lower bound from 7333371555705544704 to 7333371555601747968. Current Snapshot: MvccSnapshot[applied={T|T < 7333371555705544704}]
I20260925 22:05:14.971230 6758 mvcc.cc:205] Tried to move back new op lower bound from 7333371555705544704 to 7333371555601747968. Current Snapshot: MvccSnapshot[applied={T|T < 7333371555705544704}]
I20260925 22:05:14.974154 6236 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:05:14.974287 6236 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:05:14.975102 6236 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 64d1f9fa230447718ee1708b4c48b663: Sending DeleteTablet for 3 replicas of tablet f1ca445061924327b6ed5a886584eea3
I20260925 22:05:14.975458 6514 tablet_service.cc:1514] Processing DeleteTablet for tablet f1ca445061924327b6ed5a886584eea3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:14 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:40540
I20260925 22:05:14.975504 6374 tablet_service.cc:1514] Processing DeleteTablet for tablet f1ca445061924327b6ed5a886584eea3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:14 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:45538
I20260925 22:05:14.975481 6654 tablet_service.cc:1514] Processing DeleteTablet for tablet f1ca445061924327b6ed5a886584eea3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:14 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:41176
I20260925 22:05:14.975761 6766 tablet_replica.cc:330] T f1ca445061924327b6ed5a886584eea3 P febea1ee6a3446f39797cf6c8b74d995: stopping tablet replica
I20260925 22:05:14.975979 6767 tablet_replica.cc:330] T f1ca445061924327b6ed5a886584eea3 P 9a83e105650f4178816ee98e36b6efad: stopping tablet replica
I20260925 22:05:14.976074 6767 raft_consensus.cc:2229] T f1ca445061924327b6ed5a886584eea3 P 9a83e105650f4178816ee98e36b6efad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:05:14.975978 6236 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55692:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:05:14.976150 6236 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:05:14.976177 6767 raft_consensus.cc:2258] T f1ca445061924327b6ed5a886584eea3 P 9a83e105650f4178816ee98e36b6efad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:14.975862 6766 raft_consensus.cc:2229] T f1ca445061924327b6ed5a886584eea3 P febea1ee6a3446f39797cf6c8b74d995 [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:05:14.976387 6766 raft_consensus.cc:2258] T f1ca445061924327b6ed5a886584eea3 P febea1ee6a3446f39797cf6c8b74d995 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:14.976745 6768 tablet_replica.cc:330] T f1ca445061924327b6ed5a886584eea3 P 6457322f1e6c41ed82e7bb0068728d8e: stopping tablet replica
I20260925 22:05:14.976847 6768 raft_consensus.cc:2229] T f1ca445061924327b6ed5a886584eea3 P 6457322f1e6c41ed82e7bb0068728d8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:05:14.976951 6768 raft_consensus.cc:2258] T f1ca445061924327b6ed5a886584eea3 P 6457322f1e6c41ed82e7bb0068728d8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:14.977720 6767 ts_tablet_manager.cc:1905] T f1ca445061924327b6ed5a886584eea3 P 9a83e105650f4178816ee98e36b6efad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:14.977864 6374 tablet_service.cc:1467] Processing CreateTablet for tablet 8b8087553a9d4603b030f25f72417b13 (DEFAULT_TABLE table=test-table [id=26a9af07e6a6458091088de0f30f613e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:14.977871 6514 tablet_service.cc:1467] Processing CreateTablet for tablet 8b8087553a9d4603b030f25f72417b13 (DEFAULT_TABLE table=test-table [id=26a9af07e6a6458091088de0f30f613e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:14.977975 6374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b8087553a9d4603b030f25f72417b13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:14.978006 6514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b8087553a9d4603b030f25f72417b13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:14.979060 6745 tablet_bootstrap.cc:492] T 8b8087553a9d4603b030f25f72417b13 P febea1ee6a3446f39797cf6c8b74d995: Bootstrap starting.
I20260925 22:05:14.979396 6744 tablet_bootstrap.cc:492] T 8b8087553a9d4603b030f25f72417b13 P 9a83e105650f4178816ee98e36b6efad: Bootstrap starting.
I20260925 22:05:14.979420 6767 ts_tablet_manager.cc:1918] T f1ca445061924327b6ed5a886584eea3 P 9a83e105650f4178816ee98e36b6efad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:05:14.979490 6767 log.cc:1196] T f1ca445061924327b6ed5a886584eea3 P 9a83e105650f4178816ee98e36b6efad: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-0/wal/wals/f1ca445061924327b6ed5a886584eea3
I20260925 22:05:14.979508 6745 tablet_bootstrap.cc:654] T 8b8087553a9d4603b030f25f72417b13 P febea1ee6a3446f39797cf6c8b74d995: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:14.979681 6654 tablet_service.cc:1467] Processing CreateTablet for tablet 8b8087553a9d4603b030f25f72417b13 (DEFAULT_TABLE table=test-table [id=26a9af07e6a6458091088de0f30f613e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:14.979771 6744 tablet_bootstrap.cc:654] T 8b8087553a9d4603b030f25f72417b13 P 9a83e105650f4178816ee98e36b6efad: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:14.979792 6767 ts_tablet_manager.cc:1939] T f1ca445061924327b6ed5a886584eea3 P 9a83e105650f4178816ee98e36b6efad: Deleting consensus metadata
I20260925 22:05:14.979810 6654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b8087553a9d4603b030f25f72417b13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:14.980386 6744 tablet_bootstrap.cc:492] T 8b8087553a9d4603b030f25f72417b13 P 9a83e105650f4178816ee98e36b6efad: No bootstrap required, opened a new log
I20260925 22:05:14.980383 6766 ts_tablet_manager.cc:1905] T f1ca445061924327b6ed5a886584eea3 P febea1ee6a3446f39797cf6c8b74d995: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:14.980439 6744 ts_tablet_manager.cc:1397] T 8b8087553a9d4603b030f25f72417b13 P 9a83e105650f4178816ee98e36b6efad: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 22:05:14.980578 6744 raft_consensus.cc:348] T 8b8087553a9d4603b030f25f72417b13 P 9a83e105650f4178816ee98e36b6efad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } }
I20260925 22:05:14.980651 6744 raft_consensus.cc:374] T 8b8087553a9d4603b030f25f72417b13 P 9a83e105650f4178816ee98e36b6efad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:14.980674 6744 raft_consensus.cc:729] T 8b8087553a9d4603b030f25f72417b13 P 9a83e105650f4178816ee98e36b6efad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a83e105650f4178816ee98e36b6efad, State: Initialized, Role: FOLLOWER
I20260925 22:05:14.980729 6744 consensus_queue.cc:260] T 8b8087553a9d4603b030f25f72417b13 P 9a83e105650f4178816ee98e36b6efad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } }
I20260925 22:05:14.980818 6744 ts_tablet_manager.cc:1428] T 8b8087553a9d4603b030f25f72417b13 P 9a83e105650f4178816ee98e36b6efad: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260925 22:05:14.981233 6743 tablet_bootstrap.cc:492] T 8b8087553a9d4603b030f25f72417b13 P 6457322f1e6c41ed82e7bb0068728d8e: Bootstrap starting.
I20260925 22:05:14.981284 6223 catalog_manager.cc:4841] TS 9a83e105650f4178816ee98e36b6efad (127.5.0.65:46697): tablet f1ca445061924327b6ed5a886584eea3 (table test-table [id=0905beacf8cb40f6a6d5c9514c302f0a]) successfully deleted
I20260925 22:05:14.981638 6743 tablet_bootstrap.cc:654] T 8b8087553a9d4603b030f25f72417b13 P 6457322f1e6c41ed82e7bb0068728d8e: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:14.981832 6766 ts_tablet_manager.cc:1918] T f1ca445061924327b6ed5a886584eea3 P febea1ee6a3446f39797cf6c8b74d995: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:05:14.981895 6766 log.cc:1196] T f1ca445061924327b6ed5a886584eea3 P febea1ee6a3446f39797cf6c8b74d995: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-1/wal/wals/f1ca445061924327b6ed5a886584eea3
I20260925 22:05:14.982129 6766 ts_tablet_manager.cc:1939] T f1ca445061924327b6ed5a886584eea3 P febea1ee6a3446f39797cf6c8b74d995: Deleting consensus metadata
I20260925 22:05:14.982278 6743 tablet_bootstrap.cc:492] T 8b8087553a9d4603b030f25f72417b13 P 6457322f1e6c41ed82e7bb0068728d8e: No bootstrap required, opened a new log
I20260925 22:05:14.982342 6743 ts_tablet_manager.cc:1397] T 8b8087553a9d4603b030f25f72417b13 P 6457322f1e6c41ed82e7bb0068728d8e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 22:05:14.982511 6221 catalog_manager.cc:4841] TS febea1ee6a3446f39797cf6c8b74d995 (127.5.0.66:33975): tablet f1ca445061924327b6ed5a886584eea3 (table test-table [id=0905beacf8cb40f6a6d5c9514c302f0a]) successfully deleted
I20260925 22:05:14.982493 6743 raft_consensus.cc:348] T 8b8087553a9d4603b030f25f72417b13 P 6457322f1e6c41ed82e7bb0068728d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } }
I20260925 22:05:14.982570 6743 raft_consensus.cc:374] T 8b8087553a9d4603b030f25f72417b13 P 6457322f1e6c41ed82e7bb0068728d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:14.982606 6743 raft_consensus.cc:729] T 8b8087553a9d4603b030f25f72417b13 P 6457322f1e6c41ed82e7bb0068728d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6457322f1e6c41ed82e7bb0068728d8e, State: Initialized, Role: FOLLOWER
I20260925 22:05:14.982651 6743 consensus_queue.cc:260] T 8b8087553a9d4603b030f25f72417b13 P 6457322f1e6c41ed82e7bb0068728d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } }
I20260925 22:05:14.982743 6743 ts_tablet_manager.cc:1428] T 8b8087553a9d4603b030f25f72417b13 P 6457322f1e6c41ed82e7bb0068728d8e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 22:05:14.982837 6768 ts_tablet_manager.cc:1905] T f1ca445061924327b6ed5a886584eea3 P 6457322f1e6c41ed82e7bb0068728d8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:14.984283 6745 tablet_bootstrap.cc:492] T 8b8087553a9d4603b030f25f72417b13 P febea1ee6a3446f39797cf6c8b74d995: No bootstrap required, opened a new log
I20260925 22:05:14.984346 6745 ts_tablet_manager.cc:1397] T 8b8087553a9d4603b030f25f72417b13 P febea1ee6a3446f39797cf6c8b74d995: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260925 22:05:14.984387 6768 ts_tablet_manager.cc:1918] T f1ca445061924327b6ed5a886584eea3 P 6457322f1e6c41ed82e7bb0068728d8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:05:14.984462 6768 log.cc:1196] T f1ca445061924327b6ed5a886584eea3 P 6457322f1e6c41ed82e7bb0068728d8e: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-2/wal/wals/f1ca445061924327b6ed5a886584eea3
I20260925 22:05:14.984488 6745 raft_consensus.cc:348] T 8b8087553a9d4603b030f25f72417b13 P febea1ee6a3446f39797cf6c8b74d995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } }
I20260925 22:05:14.984567 6745 raft_consensus.cc:374] T 8b8087553a9d4603b030f25f72417b13 P febea1ee6a3446f39797cf6c8b74d995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:14.984594 6745 raft_consensus.cc:729] T 8b8087553a9d4603b030f25f72417b13 P febea1ee6a3446f39797cf6c8b74d995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: febea1ee6a3446f39797cf6c8b74d995, State: Initialized, Role: FOLLOWER
I20260925 22:05:14.984647 6745 consensus_queue.cc:260] T 8b8087553a9d4603b030f25f72417b13 P febea1ee6a3446f39797cf6c8b74d995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } }
I20260925 22:05:14.984711 6768 ts_tablet_manager.cc:1939] T f1ca445061924327b6ed5a886584eea3 P 6457322f1e6c41ed82e7bb0068728d8e: Deleting consensus metadata
I20260925 22:05:14.984731 6745 ts_tablet_manager.cc:1428] T 8b8087553a9d4603b030f25f72417b13 P febea1ee6a3446f39797cf6c8b74d995: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 22:05:14.985069 6223 catalog_manager.cc:4841] TS 6457322f1e6c41ed82e7bb0068728d8e (127.5.0.67:44357): tablet f1ca445061924327b6ed5a886584eea3 (table test-table [id=0905beacf8cb40f6a6d5c9514c302f0a]) successfully deleted
W20260925 22:05:14.985090 6441 tablet.cc:2369] T 8b8087553a9d4603b030f25f72417b13 P 9a83e105650f4178816ee98e36b6efad: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 22:05:15.050047 6721 tablet.cc:2369] T 8b8087553a9d4603b030f25f72417b13 P 6457322f1e6c41ed82e7bb0068728d8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:05:15.102700 6751 raft_consensus.cc:482] T 8b8087553a9d4603b030f25f72417b13 P 9a83e105650f4178816ee98e36b6efad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:05:15.102837 6751 raft_consensus.cc:504] T 8b8087553a9d4603b030f25f72417b13 P 9a83e105650f4178816ee98e36b6efad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } }
I20260925 22:05:15.103205 6751 leader_election.cc:288] T 8b8087553a9d4603b030f25f72417b13 P 9a83e105650f4178816ee98e36b6efad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers febea1ee6a3446f39797cf6c8b74d995 (127.5.0.66:33975), 6457322f1e6c41ed82e7bb0068728d8e (127.5.0.67:44357)
I20260925 22:05:15.107044 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:15.103428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:52895 (local address 127.5.0.66:33975)
0925 22:05:15.103498 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:15.103500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:15.103565 (+ 65us) server_negotiation.cc:407] Connection header received
0925 22:05:15.103635 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:15.103637 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:15.103646 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:15.103711 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:15.103922 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.104656 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.106146 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.106691 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.106769 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:15.106779 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:15.106941 (+ 162us) server_negotiation.cc:299] Negotiation successful
0925 22:05:15.106976 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 22:05:15.107044 6742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:15.103456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:45897 (local address 127.5.0.67:44357)
0925 22:05:15.103504 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:15.103506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:15.103608 (+ 102us) server_negotiation.cc:407] Connection header received
0925 22:05:15.103669 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:15.103670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:15.103679 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:15.103742 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:15.103919 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.104656 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.106141 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.106690 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.106769 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:15.106779 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:15.106941 (+ 162us) server_negotiation.cc:299] Negotiation successful
0925 22:05:15.106978 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 22:05:15.107074 6772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:15.103375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:33975 (local address 127.5.0.65:52895)
0925 22:05:15.103491 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:15.103504 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:15.103575 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:15.103749 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:15.103749 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:15.103754 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:15.103837 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:15.103839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.104740 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.104741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:15.106049 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:15.106052 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.106782 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.106783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:15.106858 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:15.106864 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 22:05:15.106922 (+ 58us) client_negotiation.cc:240] Negotiation successful
0925 22:05:15.106970 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":73,"thread_start_us":61,"threads_started":1}
I20260925 22:05:15.107081 6773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:15.103382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:44357 (local address 127.5.0.65:45897)
0925 22:05:15.103534 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:15.103541 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:15.103593 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:15.103749 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:15.103749 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:15.103751 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:15.103840 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:15.103841 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.104739 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.104740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:15.106049 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:15.106052 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.106781 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.106782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:15.106858 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:15.106864 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 22:05:15.106921 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 22:05:15.106970 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":85,"thread_start_us":55,"threads_started":1}
I20260925 22:05:15.107420 6674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b8087553a9d4603b030f25f72417b13" candidate_uuid: "9a83e105650f4178816ee98e36b6efad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6457322f1e6c41ed82e7bb0068728d8e" is_pre_election: true
I20260925 22:05:15.107421 6534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b8087553a9d4603b030f25f72417b13" candidate_uuid: "9a83e105650f4178816ee98e36b6efad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "febea1ee6a3446f39797cf6c8b74d995" is_pre_election: true
I20260925 22:05:15.107527 6674 raft_consensus.cc:2454] T 8b8087553a9d4603b030f25f72417b13 P 6457322f1e6c41ed82e7bb0068728d8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a83e105650f4178816ee98e36b6efad in term 0.
I20260925 22:05:15.107533 6534 raft_consensus.cc:2454] T 8b8087553a9d4603b030f25f72417b13 P febea1ee6a3446f39797cf6c8b74d995 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a83e105650f4178816ee98e36b6efad in term 0.
I20260925 22:05:15.107709 6327 leader_election.cc:302] T 8b8087553a9d4603b030f25f72417b13 P 9a83e105650f4178816ee98e36b6efad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6457322f1e6c41ed82e7bb0068728d8e, 9a83e105650f4178816ee98e36b6efad; no voters:
I20260925 22:05:15.107856 6751 raft_consensus.cc:2781] T 8b8087553a9d4603b030f25f72417b13 P 9a83e105650f4178816ee98e36b6efad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:05:15.107926 6751 raft_consensus.cc:482] T 8b8087553a9d4603b030f25f72417b13 P 9a83e105650f4178816ee98e36b6efad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:05:15.107957 6751 raft_consensus.cc:3037] T 8b8087553a9d4603b030f25f72417b13 P 9a83e105650f4178816ee98e36b6efad [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:15.108639 6751 raft_consensus.cc:504] T 8b8087553a9d4603b030f25f72417b13 P 9a83e105650f4178816ee98e36b6efad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } }
I20260925 22:05:15.108789 6751 leader_election.cc:288] T 8b8087553a9d4603b030f25f72417b13 P 9a83e105650f4178816ee98e36b6efad [CANDIDATE]: Term 1 election: Requested vote from peers febea1ee6a3446f39797cf6c8b74d995 (127.5.0.66:33975), 6457322f1e6c41ed82e7bb0068728d8e (127.5.0.67:44357)
I20260925 22:05:15.108929 6534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b8087553a9d4603b030f25f72417b13" candidate_uuid: "9a83e105650f4178816ee98e36b6efad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "febea1ee6a3446f39797cf6c8b74d995"
I20260925 22:05:15.108928 6674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b8087553a9d4603b030f25f72417b13" candidate_uuid: "9a83e105650f4178816ee98e36b6efad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6457322f1e6c41ed82e7bb0068728d8e"
I20260925 22:05:15.109001 6534 raft_consensus.cc:3037] T 8b8087553a9d4603b030f25f72417b13 P febea1ee6a3446f39797cf6c8b74d995 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:15.109001 6674 raft_consensus.cc:3037] T 8b8087553a9d4603b030f25f72417b13 P 6457322f1e6c41ed82e7bb0068728d8e [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:15.109731 6674 raft_consensus.cc:2454] T 8b8087553a9d4603b030f25f72417b13 P 6457322f1e6c41ed82e7bb0068728d8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a83e105650f4178816ee98e36b6efad in term 1.
I20260925 22:05:15.109751 6534 raft_consensus.cc:2454] T 8b8087553a9d4603b030f25f72417b13 P febea1ee6a3446f39797cf6c8b74d995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a83e105650f4178816ee98e36b6efad in term 1.
I20260925 22:05:15.109915 6325 leader_election.cc:302] T 8b8087553a9d4603b030f25f72417b13 P 9a83e105650f4178816ee98e36b6efad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a83e105650f4178816ee98e36b6efad, febea1ee6a3446f39797cf6c8b74d995; no voters:
I20260925 22:05:15.110023 6751 raft_consensus.cc:2781] T 8b8087553a9d4603b030f25f72417b13 P 9a83e105650f4178816ee98e36b6efad [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:15.110117 6751 raft_consensus.cc:686] T 8b8087553a9d4603b030f25f72417b13 P 9a83e105650f4178816ee98e36b6efad [term 1 LEADER]: Becoming Leader. State: Replica: 9a83e105650f4178816ee98e36b6efad, State: Running, Role: LEADER
I20260925 22:05:15.110183 6751 consensus_queue.cc:237] T 8b8087553a9d4603b030f25f72417b13 P 9a83e105650f4178816ee98e36b6efad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } }
I20260925 22:05:15.110890 6236 catalog_manager.cc:5495] T 8b8087553a9d4603b030f25f72417b13 P 9a83e105650f4178816ee98e36b6efad reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a83e105650f4178816ee98e36b6efad (127.5.0.65). New cstate: current_term: 1 leader_uuid: "9a83e105650f4178816ee98e36b6efad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } health_report { overall_health: UNKNOWN } } }
Sep 25 22:05:15 dist-test-slave-7mqr krb5kdc[6192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373914, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:15.155344 6781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:15.148650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:33943 (local address 127.0.0.1:55708)
0925 22:05:15.148878 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:15.148889 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:15.148957 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:15.149083 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:15.149083 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:15.149386 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:15.149434 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:15.149437 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.150065 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.150065 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:15.150576 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:15.150577 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.150897 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.150898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:15.150941 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:15.151440 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:15.151444 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:15.152849 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:15.154164 (+ 1315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:15.154170 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:15.154172 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:15.154386 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:15.154624 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:15.154625 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:15.154625 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:15.154685 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:15.154939 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:15.154946 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:15.154993 (+ 47us) client_negotiation.cc:769] Sending connection context
0925 22:05:15.155070 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 22:05:15.155208 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":131,"threads_started":1}
I20260925 22:05:15.155351 6305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:15.148782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55708 (local address 127.5.0.126:33943)
0925 22:05:15.148891 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:15.148893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:15.148956 (+ 63us) server_negotiation.cc:407] Connection header received
0925 22:05:15.149003 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:15.149004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:15.149013 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:15.149068 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:15.149502 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.149996 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.150649 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.150829 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.150884 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:15.152927 (+ 2043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:15.152928 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:15.152928 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:15.152934 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:15.154092 (+ 1158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:15.154487 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:15.154488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:15.154489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:15.154547 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:15.154763 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:15.154764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:15.154764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:15.154872 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:15.154947 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:15.155103 (+ 156us) server_negotiation.cc:299] Negotiation successful
0925 22:05:15.155271 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":90,"server-negotiator.queue_time_us":55}
I20260925 22:05:15.157373 6236 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:55708:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 22:05:15.157603 6236 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:05:15.159106 6374 tablet_service.cc:1467] Processing CreateTablet for tablet 528bbaf6b7f041aa99424633986aaaf4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fbf94b8d7a7f4b19840cfd43c0251f20]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 22:05:15.159112 6654 tablet_service.cc:1467] Processing CreateTablet for tablet 528bbaf6b7f041aa99424633986aaaf4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fbf94b8d7a7f4b19840cfd43c0251f20]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 22:05:15.159158 6514 tablet_service.cc:1467] Processing CreateTablet for tablet 528bbaf6b7f041aa99424633986aaaf4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fbf94b8d7a7f4b19840cfd43c0251f20]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 22:05:15.159273 6374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 528bbaf6b7f041aa99424633986aaaf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:15.159359 6514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 528bbaf6b7f041aa99424633986aaaf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:15.159403 6654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 528bbaf6b7f041aa99424633986aaaf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:15.160856 6745 tablet_bootstrap.cc:492] T 528bbaf6b7f041aa99424633986aaaf4 P febea1ee6a3446f39797cf6c8b74d995: Bootstrap starting.
I20260925 22:05:15.160856 6744 tablet_bootstrap.cc:492] T 528bbaf6b7f041aa99424633986aaaf4 P 9a83e105650f4178816ee98e36b6efad: Bootstrap starting.
I20260925 22:05:15.161193 6745 tablet_bootstrap.cc:654] T 528bbaf6b7f041aa99424633986aaaf4 P febea1ee6a3446f39797cf6c8b74d995: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:15.161237 6744 tablet_bootstrap.cc:654] T 528bbaf6b7f041aa99424633986aaaf4 P 9a83e105650f4178816ee98e36b6efad: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:15.161557 6743 tablet_bootstrap.cc:492] T 528bbaf6b7f041aa99424633986aaaf4 P 6457322f1e6c41ed82e7bb0068728d8e: Bootstrap starting.
I20260925 22:05:15.161733 6745 tablet_bootstrap.cc:492] T 528bbaf6b7f041aa99424633986aaaf4 P febea1ee6a3446f39797cf6c8b74d995: No bootstrap required, opened a new log
I20260925 22:05:15.161798 6745 ts_tablet_manager.cc:1397] T 528bbaf6b7f041aa99424633986aaaf4 P febea1ee6a3446f39797cf6c8b74d995: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 22:05:15.161913 6745 raft_consensus.cc:348] T 528bbaf6b7f041aa99424633986aaaf4 P febea1ee6a3446f39797cf6c8b74d995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } }
I20260925 22:05:15.161952 6743 tablet_bootstrap.cc:654] T 528bbaf6b7f041aa99424633986aaaf4 P 6457322f1e6c41ed82e7bb0068728d8e: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:15.161988 6745 raft_consensus.cc:374] T 528bbaf6b7f041aa99424633986aaaf4 P febea1ee6a3446f39797cf6c8b74d995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:15.162014 6745 raft_consensus.cc:729] T 528bbaf6b7f041aa99424633986aaaf4 P febea1ee6a3446f39797cf6c8b74d995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: febea1ee6a3446f39797cf6c8b74d995, State: Initialized, Role: FOLLOWER
I20260925 22:05:15.162065 6745 consensus_queue.cc:260] T 528bbaf6b7f041aa99424633986aaaf4 P febea1ee6a3446f39797cf6c8b74d995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } }
I20260925 22:05:15.162166 6745 ts_tablet_manager.cc:1428] T 528bbaf6b7f041aa99424633986aaaf4 P febea1ee6a3446f39797cf6c8b74d995: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:15.162205 6761 tablet_replica.cc:439] T 528bbaf6b7f041aa99424633986aaaf4 P febea1ee6a3446f39797cf6c8b74d995: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } } }
I20260925 22:05:15.162333 6761 tablet_replica.cc:442] T 528bbaf6b7f041aa99424633986aaaf4 P febea1ee6a3446f39797cf6c8b74d995: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:05:15.162544 6744 tablet_bootstrap.cc:492] T 528bbaf6b7f041aa99424633986aaaf4 P 9a83e105650f4178816ee98e36b6efad: No bootstrap required, opened a new log
I20260925 22:05:15.162595 6744 ts_tablet_manager.cc:1397] T 528bbaf6b7f041aa99424633986aaaf4 P 9a83e105650f4178816ee98e36b6efad: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 22:05:15.162750 6744 raft_consensus.cc:348] T 528bbaf6b7f041aa99424633986aaaf4 P 9a83e105650f4178816ee98e36b6efad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } }
I20260925 22:05:15.162818 6744 raft_consensus.cc:374] T 528bbaf6b7f041aa99424633986aaaf4 P 9a83e105650f4178816ee98e36b6efad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:15.162837 6744 raft_consensus.cc:729] T 528bbaf6b7f041aa99424633986aaaf4 P 9a83e105650f4178816ee98e36b6efad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a83e105650f4178816ee98e36b6efad, State: Initialized, Role: FOLLOWER
I20260925 22:05:15.162861 6743 tablet_bootstrap.cc:492] T 528bbaf6b7f041aa99424633986aaaf4 P 6457322f1e6c41ed82e7bb0068728d8e: No bootstrap required, opened a new log
I20260925 22:05:15.162889 6744 consensus_queue.cc:260] T 528bbaf6b7f041aa99424633986aaaf4 P 9a83e105650f4178816ee98e36b6efad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } }
I20260925 22:05:15.162922 6743 ts_tablet_manager.cc:1397] T 528bbaf6b7f041aa99424633986aaaf4 P 6457322f1e6c41ed82e7bb0068728d8e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 22:05:15.162985 6744 ts_tablet_manager.cc:1428] T 528bbaf6b7f041aa99424633986aaaf4 P 9a83e105650f4178816ee98e36b6efad: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:15.163038 6751 tablet_replica.cc:439] T 528bbaf6b7f041aa99424633986aaaf4 P 9a83e105650f4178816ee98e36b6efad: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } } }
I20260925 22:05:15.163059 6743 raft_consensus.cc:348] T 528bbaf6b7f041aa99424633986aaaf4 P 6457322f1e6c41ed82e7bb0068728d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } }
I20260925 22:05:15.163122 6743 raft_consensus.cc:374] T 528bbaf6b7f041aa99424633986aaaf4 P 6457322f1e6c41ed82e7bb0068728d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:15.163127 6751 tablet_replica.cc:442] T 528bbaf6b7f041aa99424633986aaaf4 P 9a83e105650f4178816ee98e36b6efad: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:05:15.163156 6743 raft_consensus.cc:729] T 528bbaf6b7f041aa99424633986aaaf4 P 6457322f1e6c41ed82e7bb0068728d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6457322f1e6c41ed82e7bb0068728d8e, State: Initialized, Role: FOLLOWER
I20260925 22:05:15.163206 6743 consensus_queue.cc:260] T 528bbaf6b7f041aa99424633986aaaf4 P 6457322f1e6c41ed82e7bb0068728d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } }
I20260925 22:05:15.163296 6743 ts_tablet_manager.cc:1428] T 528bbaf6b7f041aa99424633986aaaf4 P 6457322f1e6c41ed82e7bb0068728d8e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:15.163353 6749 tablet_replica.cc:439] T 528bbaf6b7f041aa99424633986aaaf4 P 6457322f1e6c41ed82e7bb0068728d8e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } } }
I20260925 22:05:15.163435 6749 tablet_replica.cc:442] T 528bbaf6b7f041aa99424633986aaaf4 P 6457322f1e6c41ed82e7bb0068728d8e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:05:15.172448 6749 raft_consensus.cc:482] T 528bbaf6b7f041aa99424633986aaaf4 P 6457322f1e6c41ed82e7bb0068728d8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:05:15.172518 6749 raft_consensus.cc:504] T 528bbaf6b7f041aa99424633986aaaf4 P 6457322f1e6c41ed82e7bb0068728d8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } }
I20260925 22:05:15.172816 6749 leader_election.cc:288] T 528bbaf6b7f041aa99424633986aaaf4 P 6457322f1e6c41ed82e7bb0068728d8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a83e105650f4178816ee98e36b6efad (127.5.0.65:46697), febea1ee6a3446f39797cf6c8b74d995 (127.5.0.66:33975)
I20260925 22:05:15.176402 6613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:15.172957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:46697 (local address 127.5.0.67:39399)
0925 22:05:15.173021 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:15.173034 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:15.173076 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:15.173223 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:15.173224 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:15.173226 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:15.173303 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:15.173320 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.174264 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.174265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:15.175531 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:15.175534 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.176213 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.176214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:15.176266 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:15.176270 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:05:15.176312 (+ 42us) client_negotiation.cc:240] Negotiation successful
0925 22:05:15.176344 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260925 22:05:15.176450 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:15.173055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:39399 (local address 127.5.0.65:46697)
0925 22:05:15.173097 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:15.173099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:15.173106 (+ 7us) server_negotiation.cc:407] Connection header received
0925 22:05:15.173129 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:15.173130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:15.173138 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:15.173199 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:15.173388 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.174180 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.175612 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.176137 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.176209 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:15.176219 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:15.176335 (+ 116us) server_negotiation.cc:299] Negotiation successful
0925 22:05:15.176371 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 22:05:15.176833 6394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "528bbaf6b7f041aa99424633986aaaf4" candidate_uuid: "6457322f1e6c41ed82e7bb0068728d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a83e105650f4178816ee98e36b6efad" is_pre_election: true
I20260925 22:05:15.176908 6785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:15.172963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:33975 (local address 127.5.0.67:58091)
0925 22:05:15.173113 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:15.173122 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:15.173203 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:15.173340 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:15.173341 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:15.173343 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:15.173415 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:15.173416 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.174264 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.174265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:15.175744 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:15.175746 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.176619 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.176620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:15.176725 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:15.176729 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:05:15.176787 (+ 58us) client_negotiation.cc:240] Negotiation successful
0925 22:05:15.176827 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":12,"thread_start_us":68,"threads_started":1}
I20260925 22:05:15.176900 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:15.173039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:58091 (local address 127.5.0.66:33975)
0925 22:05:15.173085 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:15.173087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:15.173186 (+ 99us) server_negotiation.cc:407] Connection header received
0925 22:05:15.173256 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:15.173257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:15.173266 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:15.173318 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:15.173474 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.174190 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.175815 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.176406 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.176472 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:15.176482 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:15.176807 (+ 325us) server_negotiation.cc:299] Negotiation successful
0925 22:05:15.176841 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 22:05:15.176934 6394 raft_consensus.cc:2454] T 528bbaf6b7f041aa99424633986aaaf4 P 9a83e105650f4178816ee98e36b6efad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6457322f1e6c41ed82e7bb0068728d8e in term 0.
I20260925 22:05:15.177069 6534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "528bbaf6b7f041aa99424633986aaaf4" candidate_uuid: "6457322f1e6c41ed82e7bb0068728d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "febea1ee6a3446f39797cf6c8b74d995" is_pre_election: true
I20260925 22:05:15.177165 6534 raft_consensus.cc:2454] T 528bbaf6b7f041aa99424633986aaaf4 P febea1ee6a3446f39797cf6c8b74d995 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6457322f1e6c41ed82e7bb0068728d8e in term 0.
I20260925 22:05:15.177292 6607 leader_election.cc:302] T 528bbaf6b7f041aa99424633986aaaf4 P 6457322f1e6c41ed82e7bb0068728d8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6457322f1e6c41ed82e7bb0068728d8e, 9a83e105650f4178816ee98e36b6efad; no voters:
I20260925 22:05:15.177433 6749 raft_consensus.cc:2781] T 528bbaf6b7f041aa99424633986aaaf4 P 6457322f1e6c41ed82e7bb0068728d8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:05:15.177496 6749 raft_consensus.cc:482] T 528bbaf6b7f041aa99424633986aaaf4 P 6457322f1e6c41ed82e7bb0068728d8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:05:15.177536 6749 raft_consensus.cc:3037] T 528bbaf6b7f041aa99424633986aaaf4 P 6457322f1e6c41ed82e7bb0068728d8e [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:15.178169 6749 raft_consensus.cc:504] T 528bbaf6b7f041aa99424633986aaaf4 P 6457322f1e6c41ed82e7bb0068728d8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } }
I20260925 22:05:15.178323 6749 leader_election.cc:288] T 528bbaf6b7f041aa99424633986aaaf4 P 6457322f1e6c41ed82e7bb0068728d8e [CANDIDATE]: Term 1 election: Requested vote from peers 9a83e105650f4178816ee98e36b6efad (127.5.0.65:46697), febea1ee6a3446f39797cf6c8b74d995 (127.5.0.66:33975)
I20260925 22:05:15.178450 6394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "528bbaf6b7f041aa99424633986aaaf4" candidate_uuid: "6457322f1e6c41ed82e7bb0068728d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a83e105650f4178816ee98e36b6efad"
I20260925 22:05:15.178478 6534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "528bbaf6b7f041aa99424633986aaaf4" candidate_uuid: "6457322f1e6c41ed82e7bb0068728d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "febea1ee6a3446f39797cf6c8b74d995"
I20260925 22:05:15.178532 6394 raft_consensus.cc:3037] T 528bbaf6b7f041aa99424633986aaaf4 P 9a83e105650f4178816ee98e36b6efad [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:15.178539 6534 raft_consensus.cc:3037] T 528bbaf6b7f041aa99424633986aaaf4 P febea1ee6a3446f39797cf6c8b74d995 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:15.179260 6394 raft_consensus.cc:2454] T 528bbaf6b7f041aa99424633986aaaf4 P 9a83e105650f4178816ee98e36b6efad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6457322f1e6c41ed82e7bb0068728d8e in term 1.
I20260925 22:05:15.179260 6534 raft_consensus.cc:2454] T 528bbaf6b7f041aa99424633986aaaf4 P febea1ee6a3446f39797cf6c8b74d995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6457322f1e6c41ed82e7bb0068728d8e in term 1.
I20260925 22:05:15.179479 6605 leader_election.cc:302] T 528bbaf6b7f041aa99424633986aaaf4 P 6457322f1e6c41ed82e7bb0068728d8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6457322f1e6c41ed82e7bb0068728d8e, febea1ee6a3446f39797cf6c8b74d995; no voters:
I20260925 22:05:15.179579 6749 raft_consensus.cc:2781] T 528bbaf6b7f041aa99424633986aaaf4 P 6457322f1e6c41ed82e7bb0068728d8e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:15.179791 6749 raft_consensus.cc:686] T 528bbaf6b7f041aa99424633986aaaf4 P 6457322f1e6c41ed82e7bb0068728d8e [term 1 LEADER]: Becoming Leader. State: Replica: 6457322f1e6c41ed82e7bb0068728d8e, State: Running, Role: LEADER
I20260925 22:05:15.179862 6749 consensus_queue.cc:237] T 528bbaf6b7f041aa99424633986aaaf4 P 6457322f1e6c41ed82e7bb0068728d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } }
I20260925 22:05:15.180246 6786 tablet_replica.cc:439] T 528bbaf6b7f041aa99424633986aaaf4 P 6457322f1e6c41ed82e7bb0068728d8e: TxnStatusTablet state changed. Reason: New leader 6457322f1e6c41ed82e7bb0068728d8e. Latest consensus state: current_term: 1 leader_uuid: "6457322f1e6c41ed82e7bb0068728d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } } }
I20260925 22:05:15.180375 6786 tablet_replica.cc:442] T 528bbaf6b7f041aa99424633986aaaf4 P 6457322f1e6c41ed82e7bb0068728d8e: This TxnStatusTablet replica's current role is: LEADER
I20260925 22:05:15.180562 6788 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 22:05:15.180651 6788 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 22:05:15.180636 6237 catalog_manager.cc:5495] T 528bbaf6b7f041aa99424633986aaaf4 P 6457322f1e6c41ed82e7bb0068728d8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6457322f1e6c41ed82e7bb0068728d8e (127.5.0.67). New cstate: current_term: 1 leader_uuid: "6457322f1e6c41ed82e7bb0068728d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } health_report { overall_health: HEALTHY } } }
I20260925 22:05:15.187297 6742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:15.184620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41192 (local address 127.5.0.67:44357)
0925 22:05:15.184659 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:15.184661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:15.184713 (+ 52us) server_negotiation.cc:407] Connection header received
0925 22:05:15.184807 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:15.184808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:15.184821 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:15.184877 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:05:15.185022 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.185459 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.186572 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.186763 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.186811 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:15.187009 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:05:15.187106 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:05:15.187157 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:15.187232 (+ 75us) server_negotiation.cc:299] Negotiation successful
0925 22:05:15.187246 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:05:15.187362 6781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:15.184572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:44357 (local address 127.0.0.1:41192)
0925 22:05:15.184660 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:15.184671 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:15.184731 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:15.184905 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:15.184907 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:15.184908 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 22:05:15.184955 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:15.184958 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.185530 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.185531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:15.186444 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:15.186447 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.186834 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.186835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:15.186905 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:15.186923 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 22:05:15.187165 (+ 242us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 22:05:15.187174 (+ 9us) client_negotiation.cc:769] Sending connection context
0925 22:05:15.187231 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 22:05:15.187280 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260925 22:05:15.188395 6394 raft_consensus.cc:1261] T 528bbaf6b7f041aa99424633986aaaf4 P 9a83e105650f4178816ee98e36b6efad [term 1 FOLLOWER]: Refusing update from remote peer 6457322f1e6c41ed82e7bb0068728d8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:05:15.188545 6534 raft_consensus.cc:1261] T 528bbaf6b7f041aa99424633986aaaf4 P febea1ee6a3446f39797cf6c8b74d995 [term 1 FOLLOWER]: Refusing update from remote peer 6457322f1e6c41ed82e7bb0068728d8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:05:15.188624 6786 consensus_queue.cc:1035] T 528bbaf6b7f041aa99424633986aaaf4 P 6457322f1e6c41ed82e7bb0068728d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:05:15.188776 6786 consensus_queue.cc:1035] T 528bbaf6b7f041aa99424633986aaaf4 P 6457322f1e6c41ed82e7bb0068728d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:05:15.189294 6761 tablet_replica.cc:439] T 528bbaf6b7f041aa99424633986aaaf4 P febea1ee6a3446f39797cf6c8b74d995: TxnStatusTablet state changed. Reason: New leader 6457322f1e6c41ed82e7bb0068728d8e. Latest consensus state: current_term: 1 leader_uuid: "6457322f1e6c41ed82e7bb0068728d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } } }
I20260925 22:05:15.189280 6751 tablet_replica.cc:439] T 528bbaf6b7f041aa99424633986aaaf4 P 9a83e105650f4178816ee98e36b6efad: TxnStatusTablet state changed. Reason: New leader 6457322f1e6c41ed82e7bb0068728d8e. Latest consensus state: current_term: 1 leader_uuid: "6457322f1e6c41ed82e7bb0068728d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } } }
I20260925 22:05:15.189368 6761 tablet_replica.cc:442] T 528bbaf6b7f041aa99424633986aaaf4 P febea1ee6a3446f39797cf6c8b74d995: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:05:15.189374 6751 tablet_replica.cc:442] T 528bbaf6b7f041aa99424633986aaaf4 P 9a83e105650f4178816ee98e36b6efad: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:05:15.189990 6786 tablet_replica.cc:439] T 528bbaf6b7f041aa99424633986aaaf4 P 6457322f1e6c41ed82e7bb0068728d8e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6457322f1e6c41ed82e7bb0068728d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } } }
I20260925 22:05:15.190062 6786 tablet_replica.cc:442] T 528bbaf6b7f041aa99424633986aaaf4 P 6457322f1e6c41ed82e7bb0068728d8e: This TxnStatusTablet replica's current role is: LEADER
I20260925 22:05:15.190179 6751 tablet_replica.cc:439] T 528bbaf6b7f041aa99424633986aaaf4 P 9a83e105650f4178816ee98e36b6efad: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6457322f1e6c41ed82e7bb0068728d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } } }
I20260925 22:05:15.190258 6751 tablet_replica.cc:442] T 528bbaf6b7f041aa99424633986aaaf4 P 9a83e105650f4178816ee98e36b6efad: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:05:15.190994 6749 tablet_replica.cc:439] T 528bbaf6b7f041aa99424633986aaaf4 P 6457322f1e6c41ed82e7bb0068728d8e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6457322f1e6c41ed82e7bb0068728d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } } }
I20260925 22:05:15.191078 6749 tablet_replica.cc:442] T 528bbaf6b7f041aa99424633986aaaf4 P 6457322f1e6c41ed82e7bb0068728d8e: This TxnStatusTablet replica's current role is: LEADER
I20260925 22:05:15.191330 6761 tablet_replica.cc:439] T 528bbaf6b7f041aa99424633986aaaf4 P febea1ee6a3446f39797cf6c8b74d995: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6457322f1e6c41ed82e7bb0068728d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a83e105650f4178816ee98e36b6efad" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46697 } } peers { permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 } } peers { permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 } } }
I20260925 22:05:15.191417 6761 tablet_replica.cc:442] T 528bbaf6b7f041aa99424633986aaaf4 P febea1ee6a3446f39797cf6c8b74d995: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:05:15.194483 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:15.191980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45554 (local address 127.5.0.65:46697)
0925 22:05:15.192016 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:15.192018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:15.192052 (+ 34us) server_negotiation.cc:407] Connection header received
0925 22:05:15.192128 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:15.192129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:15.192147 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:15.192228 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:05:15.192300 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.193007 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.193677 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.193986 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.194056 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:15.194191 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:05:15.194308 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:05:15.194376 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:15.194398 (+ 22us) server_negotiation.cc:299] Negotiation successful
0925 22:05:15.194414 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:05:15.199290 6742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:15.196915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:43443 (local address 127.5.0.67:44357)
0925 22:05:15.196970 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:15.196972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:15.197037 (+ 65us) server_negotiation.cc:407] Connection header received
0925 22:05:15.197111 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:15.197112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:15.197120 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:15.197171 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:05:15.197314 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.197770 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.198580 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.198760 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.198843 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:15.199030 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:05:15.199066 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:05:15.199138 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:15.199206 (+ 68us) server_negotiation.cc:299] Negotiation successful
0925 22:05:15.199225 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 22:05:15.199316 6796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:15.196848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:44357 (local address 127.5.0.65:43443)
0925 22:05:15.196977 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:15.196989 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:15.197053 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:15.197183 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:15.197184 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:15.197186 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 22:05:15.197239 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:15.197241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.197871 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.197872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:15.198462 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:15.198464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.198865 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.198866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:15.198933 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:15.198945 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 22:05:15.199141 (+ 196us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 22:05:15.199143 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 22:05:15.199195 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 22:05:15.199235 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":69,"threads_started":1}
I20260925 22:05:15.201606 6534 raft_consensus.cc:1261] T 8b8087553a9d4603b030f25f72417b13 P febea1ee6a3446f39797cf6c8b74d995 [term 1 FOLLOWER]: Refusing update from remote peer 9a83e105650f4178816ee98e36b6efad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:05:15.201753 6674 raft_consensus.cc:1261] T 8b8087553a9d4603b030f25f72417b13 P 6457322f1e6c41ed82e7bb0068728d8e [term 1 FOLLOWER]: Refusing update from remote peer 9a83e105650f4178816ee98e36b6efad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:05:15.201828 6751 consensus_queue.cc:1035] T 8b8087553a9d4603b030f25f72417b13 P 9a83e105650f4178816ee98e36b6efad [LEADER]: Connected to new peer: Peer: permanent_uuid: "febea1ee6a3446f39797cf6c8b74d995" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:05:15.202167 6774 consensus_queue.cc:1035] T 8b8087553a9d4603b030f25f72417b13 P 9a83e105650f4178816ee98e36b6efad [LEADER]: Connected to new peer: Peer: permanent_uuid: "6457322f1e6c41ed82e7bb0068728d8e" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:05:15.211450 6726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:15.208742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:46697 (local address 127.5.0.67:54735)
0925 22:05:15.208828 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:15.208837 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:15.208874 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:15.209026 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:15.209028 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:15.209029 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 22:05:15.209089 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:15.209090 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.209684 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.209684 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:15.210621 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:15.210624 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.211092 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.211093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:15.211150 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:15.211163 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 22:05:15.211333 (+ 170us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 22:05:15.211334 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 22:05:15.211364 (+ 30us) client_negotiation.cc:240] Negotiation successful
0925 22:05:15.211397 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260925 22:05:15.211481 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:15.208791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:54735 (local address 127.5.0.65:46697)
0925 22:05:15.208832 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:15.208834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:15.208867 (+ 33us) server_negotiation.cc:407] Connection header received
0925 22:05:15.208915 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:15.208916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:15.208925 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:15.208982 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:05:15.209148 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.209607 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.210730 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.211030 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.211092 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:15.211240 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:05:15.211273 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:05:15.211331 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:15.211370 (+ 39us) server_negotiation.cc:299] Negotiation successful
0925 22:05:15.211391 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 22:05:15.220717 6236 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:05:15.220839 6236 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:05:15.221424 6236 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 64d1f9fa230447718ee1708b4c48b663: Sending DeleteTablet for 3 replicas of tablet 8b8087553a9d4603b030f25f72417b13
I20260925 22:05:15.221675 6514 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b8087553a9d4603b030f25f72417b13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:15 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:40540
I20260925 22:05:15.221782 6766 tablet_replica.cc:330] T 8b8087553a9d4603b030f25f72417b13 P febea1ee6a3446f39797cf6c8b74d995: stopping tablet replica
I20260925 22:05:15.221773 6654 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b8087553a9d4603b030f25f72417b13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:15 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:41176
I20260925 22:05:15.221835 6766 raft_consensus.cc:2229] T 8b8087553a9d4603b030f25f72417b13 P febea1ee6a3446f39797cf6c8b74d995 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:05:15.221874 6766 raft_consensus.cc:2258] T 8b8087553a9d4603b030f25f72417b13 P febea1ee6a3446f39797cf6c8b74d995 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:15.221866 6768 tablet_replica.cc:330] T 8b8087553a9d4603b030f25f72417b13 P 6457322f1e6c41ed82e7bb0068728d8e: stopping tablet replica
I20260925 22:05:15.221906 6374 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b8087553a9d4603b030f25f72417b13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:15 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:45538
I20260925 22:05:15.221936 6768 raft_consensus.cc:2229] T 8b8087553a9d4603b030f25f72417b13 P 6457322f1e6c41ed82e7bb0068728d8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:05:15.221971 6768 raft_consensus.cc:2258] T 8b8087553a9d4603b030f25f72417b13 P 6457322f1e6c41ed82e7bb0068728d8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:15.222049 6767 tablet_replica.cc:330] T 8b8087553a9d4603b030f25f72417b13 P 9a83e105650f4178816ee98e36b6efad: stopping tablet replica
I20260925 22:05:15.222093 6767 raft_consensus.cc:2229] T 8b8087553a9d4603b030f25f72417b13 P 9a83e105650f4178816ee98e36b6efad [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:05:15.222168 6767 raft_consensus.cc:2258] T 8b8087553a9d4603b030f25f72417b13 P 9a83e105650f4178816ee98e36b6efad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:15.222282 6766 ts_tablet_manager.cc:1905] T 8b8087553a9d4603b030f25f72417b13 P febea1ee6a3446f39797cf6c8b74d995: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:15.222486 6768 ts_tablet_manager.cc:1905] T 8b8087553a9d4603b030f25f72417b13 P 6457322f1e6c41ed82e7bb0068728d8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:15.222491 6767 ts_tablet_manager.cc:1905] T 8b8087553a9d4603b030f25f72417b13 P 9a83e105650f4178816ee98e36b6efad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:15.224156 6766 ts_tablet_manager.cc:1918] T 8b8087553a9d4603b030f25f72417b13 P febea1ee6a3446f39797cf6c8b74d995: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 22:05:15.224170 6768 ts_tablet_manager.cc:1918] T 8b8087553a9d4603b030f25f72417b13 P 6457322f1e6c41ed82e7bb0068728d8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 22:05:15.224213 6766 log.cc:1196] T 8b8087553a9d4603b030f25f72417b13 P febea1ee6a3446f39797cf6c8b74d995: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-1/wal/wals/8b8087553a9d4603b030f25f72417b13
I20260925 22:05:15.224233 6768 log.cc:1196] T 8b8087553a9d4603b030f25f72417b13 P 6457322f1e6c41ed82e7bb0068728d8e: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-2/wal/wals/8b8087553a9d4603b030f25f72417b13
I20260925 22:05:15.224432 6766 ts_tablet_manager.cc:1939] T 8b8087553a9d4603b030f25f72417b13 P febea1ee6a3446f39797cf6c8b74d995: Deleting consensus metadata
I20260925 22:05:15.224457 6768 ts_tablet_manager.cc:1939] T 8b8087553a9d4603b030f25f72417b13 P 6457322f1e6c41ed82e7bb0068728d8e: Deleting consensus metadata
I20260925 22:05:15.224715 6767 ts_tablet_manager.cc:1918] T 8b8087553a9d4603b030f25f72417b13 P 9a83e105650f4178816ee98e36b6efad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 22:05:15.224758 6767 log.cc:1196] T 8b8087553a9d4603b030f25f72417b13 P 9a83e105650f4178816ee98e36b6efad: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373908987421-5121-0/minicluster-data/ts-0/wal/wals/8b8087553a9d4603b030f25f72417b13
I20260925 22:05:15.224865 6223 catalog_manager.cc:4841] TS 6457322f1e6c41ed82e7bb0068728d8e (127.5.0.67:44357): tablet 8b8087553a9d4603b030f25f72417b13 (table test-table [id=26a9af07e6a6458091088de0f30f613e]) successfully deleted
I20260925 22:05:15.224958 6767 ts_tablet_manager.cc:1939] T 8b8087553a9d4603b030f25f72417b13 P 9a83e105650f4178816ee98e36b6efad: Deleting consensus metadata
I20260925 22:05:15.224869 6221 catalog_manager.cc:4841] TS febea1ee6a3446f39797cf6c8b74d995 (127.5.0.66:33975): tablet 8b8087553a9d4603b030f25f72417b13 (table test-table [id=26a9af07e6a6458091088de0f30f613e]) successfully deleted
I20260925 22:05:15.225230 6223 catalog_manager.cc:4841] TS 9a83e105650f4178816ee98e36b6efad (127.5.0.65:46697): tablet 8b8087553a9d4603b030f25f72417b13 (table test-table [id=26a9af07e6a6458091088de0f30f613e]) successfully deleted
Sep 25 22:05:15 dist-test-slave-7mqr krb5kdc[6192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373914, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.0.65@KRBTEST.COM
I20260925 22:05:15.233675 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:15.225265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45560 (local address 127.5.0.65:46697)
0925 22:05:15.225302 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:15.225305 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:15.225551 (+ 246us) server_negotiation.cc:407] Connection header received
0925 22:05:15.225602 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:15.225603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:15.225613 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:15.225670 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:15.226266 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.226814 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.227540 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.227747 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.227832 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:15.230644 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:15.230652 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:15.230652 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:15.230659 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:15.232417 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:15.232792 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:15.232795 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:15.232795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:15.232868 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:15.233074 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:15.233075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:15.233075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:15.233225 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:15.233366 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:15.233445 (+ 79us) server_negotiation.cc:299] Negotiation successful
0925 22:05:15.233524 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260925 22:05:15.233996 6434 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45560
I20260925 22:05:15.241452 6305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:15.234893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55718 (local address 127.5.0.126:33943)
0925 22:05:15.234945 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:15.234948 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:15.235530 (+ 582us) server_negotiation.cc:407] Connection header received
0925 22:05:15.235632 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:15.235634 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:15.235644 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:15.235672 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:15.236257 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.236738 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.237414 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:15.237629 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:15.237713 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:15.238824 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:15.238825 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:15.238826 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:15.238833 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:15.240309 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:15.240643 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:15.240644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:15.240644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:15.240697 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:15.240900 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:15.240901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:15.240901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:15.241041 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:15.241146 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:15.241271 (+ 125us) server_negotiation.cc:299] Negotiation successful
0925 22:05:15.241343 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260925 22:05:15.241652 6236 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55718
I20260925 22:05:15.242218 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 6309
I20260925 22:05:15.249675 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 6449
I20260925 22:05:15.256196 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 6589
I20260925 22:05:15.261411 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 6205
2026-09-25T22:05:15Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2241 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:05:15 dist-test-slave-7mqr krb5kdc[6809](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:05:15 dist-test-slave-7mqr krb5kdc[6809](info): set up 2 sockets
Sep 25 22:05:15 dist-test-slave-7mqr krb5kdc[6809](info): commencing operation
krb5kdc: starting...
W20260925 22:05:16.303576 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:05:16 dist-test-slave-7mqr krb5kdc[6809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:05:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:05:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.0.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
I20260925 22:05:16.416780 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:34405
--webserver_interface=127.5.0.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:41017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:34405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:16.496374 6822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:16.496582 6822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:16.496603 6822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:16.497769 6822 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:05:16.497814 6822 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 22:05:16.497828 6822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:16.497877 6822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:16.497902 6822 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:05:16.497924 6822 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:05:16.499401 6822 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:41017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:34405
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:34405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:16.499596 6822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:16.499851 6822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:16.504730 6827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:16.504853 6822 server_base.cc:1034] running on GCE node
W20260925 22:05:16.504690 6830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:16.504686 6828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:16.505296 6822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:16.505630 6822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:16.506803 6822 hybrid_clock.cc:648] HybridClock initialized: now 1790373916506801 us; error 43 us; skew 500 ppm
Sep 25 22:05:16 dist-test-slave-7mqr krb5kdc[6809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373916, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:16.509405 6822 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:05:16.509805 6822 webserver.cc:460] Webserver started at http://127.5.0.126:42075/ using document root <none> and password file <none>
I20260925 22:05:16.510010 6822 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:16.510051 6822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:16.510144 6822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:16.511185 6822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "196ba701dd2e4bb9a12bc02d704b73ce"
format_stamp: "Formatted at 2026-09-25 22:05:16 on dist-test-slave-7mqr"
server_key: "c51597b7ffaa9797a3ed47dfe4c0ad41"
server_key_iv: "89849ca2e04c87826febb2e0fb461694"
server_key_version: "encryptionkey@0"
I20260925 22:05:16.511482 6822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "196ba701dd2e4bb9a12bc02d704b73ce"
format_stamp: "Formatted at 2026-09-25 22:05:16 on dist-test-slave-7mqr"
server_key: "c51597b7ffaa9797a3ed47dfe4c0ad41"
server_key_iv: "89849ca2e04c87826febb2e0fb461694"
server_key_version: "encryptionkey@0"
I20260925 22:05:16.512792 6822 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:05:16.513511 6837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:16.513725 6822 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:05:16.513818 6822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "196ba701dd2e4bb9a12bc02d704b73ce"
format_stamp: "Formatted at 2026-09-25 22:05:16 on dist-test-slave-7mqr"
server_key: "c51597b7ffaa9797a3ed47dfe4c0ad41"
server_key_iv: "89849ca2e04c87826febb2e0fb461694"
server_key_version: "encryptionkey@0"
I20260925 22:05:16.513871 6822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:16.523451 6822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:16.525671 6822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:16.525776 6822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:16.530071 6822 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:34405
I20260925 22:05:16.530144 6899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:34405 every 8 connection(s)
I20260925 22:05:16.530467 6822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:05:16.531046 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 6822
I20260925 22:05:16.531069 6900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:16.531147 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:05:16.531316 5121 external_mini_cluster.cc:1428] Setting key ef3fbd9dd580bdbd89c76df5ceea876b
I20260925 22:05:16.534163 6900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 196ba701dd2e4bb9a12bc02d704b73ce: Bootstrap starting.
I20260925 22:05:16.534932 6900 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 196ba701dd2e4bb9a12bc02d704b73ce: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:16.535215 6900 log.cc:824] T 00000000000000000000000000000000 P 196ba701dd2e4bb9a12bc02d704b73ce: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:16.535933 6900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 196ba701dd2e4bb9a12bc02d704b73ce: No bootstrap required, opened a new log
Sep 25 22:05:16 dist-test-slave-7mqr krb5kdc[6809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:16.538283 6900 raft_consensus.cc:348] T 00000000000000000000000000000000 P 196ba701dd2e4bb9a12bc02d704b73ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "196ba701dd2e4bb9a12bc02d704b73ce" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34405 } }
I20260925 22:05:16.538444 6900 raft_consensus.cc:374] T 00000000000000000000000000000000 P 196ba701dd2e4bb9a12bc02d704b73ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:16.538491 6900 raft_consensus.cc:729] T 00000000000000000000000000000000 P 196ba701dd2e4bb9a12bc02d704b73ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 196ba701dd2e4bb9a12bc02d704b73ce, State: Initialized, Role: FOLLOWER
I20260925 22:05:16.538646 6900 consensus_queue.cc:260] T 00000000000000000000000000000000 P 196ba701dd2e4bb9a12bc02d704b73ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "196ba701dd2e4bb9a12bc02d704b73ce" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34405 } }
I20260925 22:05:16.538729 6900 raft_consensus.cc:388] T 00000000000000000000000000000000 P 196ba701dd2e4bb9a12bc02d704b73ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:05:16.538774 6900 raft_consensus.cc:482] T 00000000000000000000000000000000 P 196ba701dd2e4bb9a12bc02d704b73ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:05:16.538821 6900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 196ba701dd2e4bb9a12bc02d704b73ce [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:16.539367 6900 raft_consensus.cc:504] T 00000000000000000000000000000000 P 196ba701dd2e4bb9a12bc02d704b73ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "196ba701dd2e4bb9a12bc02d704b73ce" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34405 } }
I20260925 22:05:16.539487 6900 leader_election.cc:302] T 00000000000000000000000000000000 P 196ba701dd2e4bb9a12bc02d704b73ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 196ba701dd2e4bb9a12bc02d704b73ce; no voters:
I20260925 22:05:16.539667 6900 leader_election.cc:288] T 00000000000000000000000000000000 P 196ba701dd2e4bb9a12bc02d704b73ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:05:16.539742 6905 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 196ba701dd2e4bb9a12bc02d704b73ce [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:16.539932 6905 raft_consensus.cc:686] T 00000000000000000000000000000000 P 196ba701dd2e4bb9a12bc02d704b73ce [term 1 LEADER]: Becoming Leader. State: Replica: 196ba701dd2e4bb9a12bc02d704b73ce, State: Running, Role: LEADER
I20260925 22:05:16.539965 6900 sys_catalog.cc:564] T 00000000000000000000000000000000 P 196ba701dd2e4bb9a12bc02d704b73ce [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:05:16.540038 6905 consensus_queue.cc:237] T 00000000000000000000000000000000 P 196ba701dd2e4bb9a12bc02d704b73ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "196ba701dd2e4bb9a12bc02d704b73ce" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34405 } }
I20260925 22:05:16.540359 6905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 196ba701dd2e4bb9a12bc02d704b73ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 196ba701dd2e4bb9a12bc02d704b73ce. Latest consensus state: current_term: 1 leader_uuid: "196ba701dd2e4bb9a12bc02d704b73ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "196ba701dd2e4bb9a12bc02d704b73ce" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34405 } } }
I20260925 22:05:16.540503 6905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 196ba701dd2e4bb9a12bc02d704b73ce [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:16.541440 6914 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:05:16.541888 6906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 196ba701dd2e4bb9a12bc02d704b73ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "196ba701dd2e4bb9a12bc02d704b73ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "196ba701dd2e4bb9a12bc02d704b73ce" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34405 } } }
I20260925 22:05:16.542002 6914 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:05:16.542203 6906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 196ba701dd2e4bb9a12bc02d704b73ce [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:16.543078 6903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:16.531816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50416 (local address 127.5.0.126:34405)
0925 22:05:16.532033 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:16.532040 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:16.532064 (+ 24us) server_negotiation.cc:407] Connection header received
0925 22:05:16.532500 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:16.532506 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:16.532557 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:16.532669 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:16.533445 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:16.534339 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:16.535420 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:16.535811 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:16.535948 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:16.538430 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:16.538440 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:16.538441 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:16.538451 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:16.540695 (+ 2244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:16.541290 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:16.541294 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:16.541294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:16.541412 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:16.541700 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:16.541700 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:16.541701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:16.542034 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:16.542595 (+ 561us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:16.542667 (+ 72us) server_negotiation.cc:299] Negotiation successful
0925 22:05:16.542775 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":104,"threads_started":1}
I20260925 22:05:16.543795 6914 catalog_manager.cc:1347] Generated new cluster ID: 4714ac7d957e412d98c89dcdcc96e3ed
I20260925 22:05:16.543846 6914 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:05:16.552345 6914 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:05:16.552865 6914 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:05:16.557283 6914 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 196ba701dd2e4bb9a12bc02d704b73ce: Generated new TSK 0
I20260925 22:05:16.557456 6914 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.0.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
I20260925 22:05:16.588752 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.65:0
--local_ip_for_outbound_sockets=127.5.0.65
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:34405
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:41017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:16.667843 6926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:16.668051 6926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:16.668075 6926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:16.669135 6926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:16.669166 6926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:16.669186 6926 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 22:05:16.669198 6926 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 22:05:16.669214 6926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.65
I20260925 22:05:16.670544 6926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:41017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.0.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.0.126:34405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.65
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:16.670717 6926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:16.670877 6926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:16.675568 6932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:16.675592 6926 server_base.cc:1034] running on GCE node
W20260925 22:05:16.675577 6934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:16.675577 6931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:16.676092 6926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:16.676331 6926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:16.677496 6926 hybrid_clock.cc:648] HybridClock initialized: now 1790373916677482 us; error 28 us; skew 500 ppm
Sep 25 22:05:16 dist-test-slave-7mqr krb5kdc[6809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373916, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:16.680068 6926 init.cc:377] Logged in from keytab as kudu/127.5.0.65@KRBTEST.COM (short username kudu)
I20260925 22:05:16.680502 6926 webserver.cc:460] Webserver started at http://127.5.0.65:34407/ using document root <none> and password file <none>
I20260925 22:05:16.680737 6926 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:16.680801 6926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:16.680933 6926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:16.681967 6926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-0/data/instance:
uuid: "874ee8b794ef4086a9601b6a2df8dee6"
format_stamp: "Formatted at 2026-09-25 22:05:16 on dist-test-slave-7mqr"
server_key: "e75a34d6512d99d6af0238ae7c48180a"
server_key_iv: "6424a1f10ffdfa6c055a9b11f6ce7b28"
server_key_version: "encryptionkey@0"
I20260925 22:05:16.682392 6926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance:
uuid: "874ee8b794ef4086a9601b6a2df8dee6"
format_stamp: "Formatted at 2026-09-25 22:05:16 on dist-test-slave-7mqr"
server_key: "e75a34d6512d99d6af0238ae7c48180a"
server_key_iv: "6424a1f10ffdfa6c055a9b11f6ce7b28"
server_key_version: "encryptionkey@0"
I20260925 22:05:16.684094 6926 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260925 22:05:16.684859 6941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:16.685040 6926 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:05:16.685151 6926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-0/wal
uuid: "874ee8b794ef4086a9601b6a2df8dee6"
format_stamp: "Formatted at 2026-09-25 22:05:16 on dist-test-slave-7mqr"
server_key: "e75a34d6512d99d6af0238ae7c48180a"
server_key_iv: "6424a1f10ffdfa6c055a9b11f6ce7b28"
server_key_version: "encryptionkey@0"
I20260925 22:05:16.685235 6926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:16.701858 6926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:16.704341 6926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:16.704504 6926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:16.705057 6926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:16.705119 6926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:16.705192 6926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:16.705240 6926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 22:05:16 dist-test-slave-7mqr krb5kdc[6809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373916, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:16.712519 6926 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.65:36769
I20260925 22:05:16.712841 7056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.65:36769 every 8 connection(s)
I20260925 22:05:16.712932 6926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
I20260925 22:05:16.713586 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 6926
I20260925 22:05:16.713675 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance
I20260925 22:05:16.713864 5121 external_mini_cluster.cc:1428] Setting key cd701efc7b07b3fc8528128456623220
I20260925 22:05:16.715518 6950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:16.705446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34405 (local address 127.5.0.65:48767)
0925 22:05:16.705604 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:16.705639 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:16.706160 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:16.706427 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:16.706431 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:16.706722 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:16.706935 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:16.706940 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:16.707645 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:16.707646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:16.708586 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:16.708589 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:16.708999 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:16.709000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:16.709178 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:16.709740 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:16.709752 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:16.712226 (+ 2474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:16.714182 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:16.714188 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:16.714190 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:16.714464 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:16.714694 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:16.714695 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:16.714695 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:16.714787 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:16.715061 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:16.715063 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:16.715178 (+ 115us) client_negotiation.cc:769] Sending connection context
0925 22:05:16.715256 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 22:05:16.715367 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":76,"threads_started":1}
I20260925 22:05:16.715531 6903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:16.705447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:48767 (local address 127.5.0.126:34405)
0925 22:05:16.705489 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:16.705492 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:16.705735 (+ 243us) server_negotiation.cc:407] Connection header received
0925 22:05:16.706280 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:16.706281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:16.706294 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:16.706352 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:16.707036 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:16.707546 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:16.708682 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:16.708898 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:16.708986 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:16.712360 (+ 3374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:16.712361 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:16.712362 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:16.712371 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:16.714116 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:16.714538 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:16.714539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:16.714539 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:16.714628 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:16.714849 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:16.714849 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:16.714850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:16.714996 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:16.715075 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:16.715300 (+ 225us) server_negotiation.cc:299] Negotiation successful
0925 22:05:16.715393 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 22:05:16.716380 7057 heartbeater.cc:347] Connected to a master server at 127.5.0.126:34405
I20260925 22:05:16.716472 7057 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:16.716684 7057 heartbeater.cc:511] Master 127.5.0.126:34405 requested a full tablet report, sending...
I20260925 22:05:16.717180 6853 ts_manager.cc:194] Registered new tserver with Master: 874ee8b794ef4086a9601b6a2df8dee6 (127.5.0.65:36769)
I20260925 22:05:16.717896 6853 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.65@KRBTEST.COM'} at 127.5.0.65:48767
WARNING: no policy specified for kudu/127.5.0.66@KRBTEST.COM; defaulting to no policy
I20260925 22:05:16.725596 7064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:16.718207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34405 (local address 127.5.0.65:57953)
0925 22:05:16.719807 (+ 1600us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:16.719820 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:16.719923 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:16.720051 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:16.720052 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:16.720260 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:16.720327 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:16.720329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:16.720936 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:16.720938 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:16.721871 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:16.721874 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:16.722204 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:16.722205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:16.722293 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:16.722696 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:16.722702 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:16.723091 (+ 389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:16.724768 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:16.724769 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:16.724769 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:16.724926 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:16.725084 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:16.725085 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:16.725085 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:16.725163 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:16.725367 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:16.725367 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:16.725401 (+ 34us) client_negotiation.cc:769] Sending connection context
0925 22:05:16.725455 (+ 54us) client_negotiation.cc:240] Negotiation successful
0925 22:05:16.725526 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1569,"thread_start_us":1517,"threads_started":1}
I20260925 22:05:16.725601 6903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:16.718273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:57953 (local address 127.5.0.126:34405)
0925 22:05:16.718314 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:16.718315 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:16.719925 (+ 1610us) server_negotiation.cc:407] Connection header received
0925 22:05:16.719987 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:16.719988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:16.719997 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:16.720044 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:16.720399 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:16.720840 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:16.721956 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:16.722130 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:16.722183 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:16.723177 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:16.723178 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:16.723178 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:16.723184 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:16.724696 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:16.724989 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:16.724990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:16.724990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:16.725033 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:16.725221 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:16.725221 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:16.725222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:16.725311 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:16.725359 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:16.725472 (+ 113us) server_negotiation.cc:299] Negotiation successful
0925 22:05:16.725546 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
I20260925 22:05:16.742262 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.66:0
--local_ip_for_outbound_sockets=127.5.0.66
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:34405
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:41017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:16.822057 7066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:16.822292 7066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:16.822330 7066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:16.823469 7066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:16.823536 7066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:16.823570 7066 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 22:05:16.823599 7066 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 22:05:16.823638 7066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.66
I20260925 22:05:16.825073 7066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:41017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.0.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.0.126:34405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.66
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:16.825343 7066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:16.825560 7066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:16.830555 7074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:16.830590 7071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:16.830595 7072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:16.830821 7066 server_base.cc:1034] running on GCE node
I20260925 22:05:16.831002 7066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:16.831193 7066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:16.832332 7066 hybrid_clock.cc:648] HybridClock initialized: now 1790373916832321 us; error 27 us; skew 500 ppm
Sep 25 22:05:16 dist-test-slave-7mqr krb5kdc[6809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373916, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:16.834707 7066 init.cc:377] Logged in from keytab as kudu/127.5.0.66@KRBTEST.COM (short username kudu)
I20260925 22:05:16.835115 7066 webserver.cc:460] Webserver started at http://127.5.0.66:45089/ using document root <none> and password file <none>
I20260925 22:05:16.835397 7066 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:16.835466 7066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:16.835604 7066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:16.836819 7066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-1/data/instance:
uuid: "10148bd29f0445239c5b4e9793900f5f"
format_stamp: "Formatted at 2026-09-25 22:05:16 on dist-test-slave-7mqr"
server_key: "62282b83a458945891694b70c317a70b"
server_key_iv: "4253a606a9e13f0bbdffb1835a8c12f8"
server_key_version: "encryptionkey@0"
I20260925 22:05:16.837194 7066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance:
uuid: "10148bd29f0445239c5b4e9793900f5f"
format_stamp: "Formatted at 2026-09-25 22:05:16 on dist-test-slave-7mqr"
server_key: "62282b83a458945891694b70c317a70b"
server_key_iv: "4253a606a9e13f0bbdffb1835a8c12f8"
server_key_version: "encryptionkey@0"
I20260925 22:05:16.838486 7066 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 22:05:16.839425 7081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:16.839634 7066 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:05:16.839733 7066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-1/wal
uuid: "10148bd29f0445239c5b4e9793900f5f"
format_stamp: "Formatted at 2026-09-25 22:05:16 on dist-test-slave-7mqr"
server_key: "62282b83a458945891694b70c317a70b"
server_key_iv: "4253a606a9e13f0bbdffb1835a8c12f8"
server_key_version: "encryptionkey@0"
I20260925 22:05:16.839792 7066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:16.872227 7066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:16.874109 7066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:16.874220 7066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:16.874684 7066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:16.874724 7066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:16.874750 7066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:16.874788 7066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:16.881536 7066 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.66:33273
I20260925 22:05:16.881600 7196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.66:33273 every 8 connection(s)
I20260925 22:05:16.881909 7066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
Sep 25 22:05:16 dist-test-slave-7mqr krb5kdc[6809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373916, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:16.885390 6903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:16.875228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:40151 (local address 127.5.0.126:34405)
0925 22:05:16.875302 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:16.875304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:16.875517 (+ 213us) server_negotiation.cc:407] Connection header received
0925 22:05:16.876094 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:16.876095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:16.876110 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:16.876202 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:16.877044 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:16.877848 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:16.879555 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:16.879792 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:16.879851 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:16.882611 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:16.882613 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:16.882613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:16.882620 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:16.884041 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:16.884424 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:16.884425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:16.884426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:16.884477 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:16.884701 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:16.884702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:16.884702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:16.884848 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:16.884904 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:16.885246 (+ 342us) server_negotiation.cc:299] Negotiation successful
0925 22:05:16.885325 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 22:05:16.885615 7090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:16.875206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34405 (local address 127.5.0.66:40151)
0925 22:05:16.875380 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:16.875420 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:16.875978 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:16.876297 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:16.876302 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:16.876665 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:16.876955 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:16.876962 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:16.877953 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:16.877954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:16.879456 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:16.879460 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:16.879908 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:16.879909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:16.880061 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:16.880652 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:16.880663 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:16.882523 (+ 1860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:16.884139 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:16.884145 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:16.884147 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:16.884360 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:16.884532 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:16.884533 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:16.884533 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:16.884634 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:16.884931 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:16.884938 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:16.885075 (+ 137us) client_negotiation.cc:769] Sending connection context
0925 22:05:16.885193 (+ 118us) client_negotiation.cc:240] Negotiation successful
0925 22:05:16.885335 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":102,"threads_started":1}
I20260925 22:05:16.886164 7197 heartbeater.cc:347] Connected to a master server at 127.5.0.126:34405
I20260925 22:05:16.886253 7197 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:16.886456 7197 heartbeater.cc:511] Master 127.5.0.126:34405 requested a full tablet report, sending...
I20260925 22:05:16.886900 6853 ts_manager.cc:194] Registered new tserver with Master: 10148bd29f0445239c5b4e9793900f5f (127.5.0.66:33273)
I20260925 22:05:16.887290 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 7066
I20260925 22:05:16.887394 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance
I20260925 22:05:16.887459 6853 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.66@KRBTEST.COM'} at 127.5.0.66:40151
I20260925 22:05:16.887631 5121 external_mini_cluster.cc:1428] Setting key 480201a98e72be72bb43615ae93d8d21
I20260925 22:05:16.894315 6903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:16.887822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:55995 (local address 127.5.0.126:34405)
0925 22:05:16.887875 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:16.887877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:16.887884 (+ 7us) server_negotiation.cc:407] Connection header received
0925 22:05:16.887934 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:16.887935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:16.887944 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:16.887991 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:16.888399 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:16.889226 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:16.890139 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:16.890427 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:16.890503 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:16.891580 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:16.891581 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:16.891581 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:16.891587 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:16.893149 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:16.893570 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:16.893571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:16.893572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:16.893627 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:16.893840 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:16.893841 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:16.893841 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:16.893969 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:16.894022 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:16.894200 (+ 178us) server_negotiation.cc:299] Negotiation successful
0925 22:05:16.894258 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 22:05:16.894411 7203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:16.887370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34405 (local address 127.5.0.66:55995)
0925 22:05:16.887538 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:16.887550 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:16.887609 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:16.888075 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:16.888076 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:16.888270 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:16.888329 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:16.888331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:16.889318 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:16.889319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:16.890038 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:16.890041 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:16.890521 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:16.890522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:16.890603 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:16.891054 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:16.891059 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:16.891466 (+ 407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:16.893237 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:16.893238 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:16.893239 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:16.893479 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:16.893700 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:16.893701 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:16.893701 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:16.893772 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:16.894044 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:16.894045 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:16.894096 (+ 51us) client_negotiation.cc:769] Sending connection context
0925 22:05:16.894183 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 22:05:16.894292 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":86,"threads_started":1}
WARNING: no policy specified for kudu/127.5.0.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
I20260925 22:05:16.914234 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.67:0
--local_ip_for_outbound_sockets=127.5.0.67
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:34405
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:41017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:16.995078 7206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:16.995288 7206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:16.995312 7206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:16.996552 7206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:16.996596 7206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:16.996618 7206 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 22:05:16.996630 7206 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 22:05:16.996675 7206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.67
I20260925 22:05:16.998134 7206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:41017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.0.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.0.126:34405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.67
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:16.998386 7206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:16.998624 7206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:17.003489 7211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:17.003523 7214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:17.003551 7212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:17.003881 7206 server_base.cc:1034] running on GCE node
I20260925 22:05:17.004132 7206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:17.004439 7206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:17.005606 7206 hybrid_clock.cc:648] HybridClock initialized: now 1790373917005597 us; error 42 us; skew 500 ppm
Sep 25 22:05:17 dist-test-slave-7mqr krb5kdc[6809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373917, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:17.008104 7206 init.cc:377] Logged in from keytab as kudu/127.5.0.67@KRBTEST.COM (short username kudu)
I20260925 22:05:17.008596 7206 webserver.cc:460] Webserver started at http://127.5.0.67:41649/ using document root <none> and password file <none>
I20260925 22:05:17.008849 7206 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:17.008919 7206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:17.009079 7206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:17.010135 7206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-2/data/instance:
uuid: "38876e68cdfa46158375464729ee36f4"
format_stamp: "Formatted at 2026-09-25 22:05:17 on dist-test-slave-7mqr"
server_key: "77675b1009d5245d88eb5329a9a739fa"
server_key_iv: "b66f6b1e7bcd8b5c0379a8df2ac8c814"
server_key_version: "encryptionkey@0"
I20260925 22:05:17.010501 7206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance:
uuid: "38876e68cdfa46158375464729ee36f4"
format_stamp: "Formatted at 2026-09-25 22:05:17 on dist-test-slave-7mqr"
server_key: "77675b1009d5245d88eb5329a9a739fa"
server_key_iv: "b66f6b1e7bcd8b5c0379a8df2ac8c814"
server_key_version: "encryptionkey@0"
I20260925 22:05:17.011813 7206 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 22:05:17.012543 7221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:17.012702 7206 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:05:17.012804 7206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-2/wal
uuid: "38876e68cdfa46158375464729ee36f4"
format_stamp: "Formatted at 2026-09-25 22:05:17 on dist-test-slave-7mqr"
server_key: "77675b1009d5245d88eb5329a9a739fa"
server_key_iv: "b66f6b1e7bcd8b5c0379a8df2ac8c814"
server_key_version: "encryptionkey@0"
I20260925 22:05:17.012892 7206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:17.045671 7206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:17.047591 7206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:17.047761 7206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:17.048393 7206 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:17.048450 7206 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:17.048502 7206 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:17.048521 7206 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 22:05:17 dist-test-slave-7mqr krb5kdc[6809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373917, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:17.055694 7206 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.67:37045
I20260925 22:05:17.056105 7206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
I20260925 22:05:17.055780 7336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.67:37045 every 8 connection(s)
I20260925 22:05:17.058614 6903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:17.048773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:48923 (local address 127.5.0.126:34405)
0925 22:05:17.048834 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:17.048836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:17.049045 (+ 209us) server_negotiation.cc:407] Connection header received
0925 22:05:17.049646 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:17.049647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:17.049660 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:17.049711 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:17.050550 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.051396 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.052599 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.052814 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.052902 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:17.055513 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:17.055514 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:17.055515 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:17.055523 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:17.057197 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:17.057640 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:17.057641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:17.057641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:17.057709 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:17.057979 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:17.057980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:17.057980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:17.058101 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:17.058178 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:17.058445 (+ 267us) server_negotiation.cc:299] Negotiation successful
0925 22:05:17.058523 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 22:05:17.058815 7230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:17.048777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34405 (local address 127.5.0.67:48923)
0925 22:05:17.048928 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:17.048963 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:17.049528 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:17.049783 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:17.049786 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:17.050147 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:17.050438 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:17.050444 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.051496 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.051497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:17.052500 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:17.052503 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.052923 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.052924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:17.053052 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:17.053617 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:17.053628 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:17.055403 (+ 1775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:17.057301 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:17.057307 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:17.057309 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:17.057544 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:17.057795 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:17.057796 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:17.057796 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:17.057904 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:17.058169 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:17.058175 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:17.058314 (+ 139us) client_negotiation.cc:769] Sending connection context
0925 22:05:17.058423 (+ 109us) client_negotiation.cc:240] Negotiation successful
0925 22:05:17.058575 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
I20260925 22:05:17.059268 7337 heartbeater.cc:347] Connected to a master server at 127.5.0.126:34405
I20260925 22:05:17.059345 7337 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:17.059520 7337 heartbeater.cc:511] Master 127.5.0.126:34405 requested a full tablet report, sending...
I20260925 22:05:17.059547 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 7206
I20260925 22:05:17.059676 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance
I20260925 22:05:17.059859 5121 external_mini_cluster.cc:1428] Setting key 5d4d713a23ff0e77a2c17903838d13d0
I20260925 22:05:17.059875 6854 ts_manager.cc:194] Registered new tserver with Master: 38876e68cdfa46158375464729ee36f4 (127.5.0.67:37045)
I20260925 22:05:17.060412 6854 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.67@KRBTEST.COM'} at 127.5.0.67:48923
I20260925 22:05:17.060868 5121 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:05:17.065825 7343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:17.060266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34405 (local address 127.5.0.67:35121)
0925 22:05:17.060408 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:17.060419 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:17.060495 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:17.060669 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:17.060670 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:17.060881 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:17.060947 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:17.060949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.061676 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.061677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:17.062160 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:17.062161 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.062522 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.062523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:17.062602 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:17.063001 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:17.063004 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:17.063256 (+ 252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:17.064808 (+ 1552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:17.064809 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:17.064810 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:17.064976 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:17.065186 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:17.065187 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:17.065187 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:17.065243 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:17.065491 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:17.065492 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:17.065532 (+ 40us) client_negotiation.cc:769] Sending connection context
0925 22:05:17.065622 (+ 90us) client_negotiation.cc:240] Negotiation successful
0925 22:05:17.065727 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":95,"threads_started":1}
I20260925 22:05:17.065811 6903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:17.060448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:35121 (local address 127.5.0.126:34405)
0925 22:05:17.060525 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:17.060526 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:17.060535 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:05:17.060554 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:17.060554 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:17.060564 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:17.060615 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:17.061035 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.061583 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.062245 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.062448 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.062523 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:17.063342 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:17.063364 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:17.063365 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:17.063374 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:17.064708 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:17.065061 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:17.065062 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:17.065063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:17.065120 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:17.065315 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:17.065316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:17.065316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:17.065424 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:17.065499 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:17.065649 (+ 150us) server_negotiation.cc:299] Negotiation successful
0925 22:05:17.065718 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
Sep 25 22:05:17 dist-test-slave-7mqr krb5kdc[6809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 22:05:17 dist-test-slave-7mqr krb5kdc[6809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:17.082847 6903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:17.076619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50422 (local address 127.5.0.126:34405)
0925 22:05:17.076671 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:17.076673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:17.076682 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:05:17.076708 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:17.076709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:17.076717 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:17.076764 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:17.077290 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.077743 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.078414 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.078581 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.078627 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:17.080784 (+ 2157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:17.080786 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:17.080786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:17.080792 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:17.081941 (+ 1149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:17.082253 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:17.082253 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:17.082254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:17.082300 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:17.082456 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:17.082457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:17.082457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:17.082547 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:17.082617 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:17.082726 (+ 109us) server_negotiation.cc:299] Negotiation successful
0925 22:05:17.082784 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 22:05:17.084501 6853 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50422:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:05:17.086146 6853 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:05:17.092650 7358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:17.088605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:36769 (local address 127.0.0.1:44022)
0925 22:05:17.088934 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:17.088976 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:17.089049 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:17.089265 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:17.089267 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:17.089270 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:17.089337 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:17.089338 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.090365 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.090366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:17.091492 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:17.091495 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.092379 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.092380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:17.092457 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:17.092468 (+ 11us) client_negotiation.cc:769] Sending connection context
0925 22:05:17.092531 (+ 63us) client_negotiation.cc:240] Negotiation successful
0925 22:05:17.092579 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":74,"threads_started":1}
I20260925 22:05:17.092983 7357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:17.088763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44022 (local address 127.5.0.65:36769)
0925 22:05:17.088933 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:17.088937 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:17.089038 (+ 101us) server_negotiation.cc:407] Connection header received
0925 22:05:17.089142 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:17.089145 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:17.089159 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:17.089260 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:17.089420 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.090285 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.091590 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.092223 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.092290 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:17.092346 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:17.092832 (+ 486us) server_negotiation.cc:299] Negotiation successful
0925 22:05:17.092880 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":80,"threads_started":1}
I20260925 22:05:17.093252 7356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:17.088710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37280 (local address 127.5.0.67:37045)
0925 22:05:17.088888 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:17.088894 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:17.089270 (+ 376us) server_negotiation.cc:407] Connection header received
0925 22:05:17.089370 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:17.089374 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:17.089392 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:17.089470 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:17.089585 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.090687 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.092111 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.092821 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.092883 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:17.092939 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:17.093135 (+ 196us) server_negotiation.cc:299] Negotiation successful
0925 22:05:17.093174 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":90,"threads_started":1}
I20260925 22:05:17.093299 7355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:17.088817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:33273 (local address 127.0.0.1:42444)
0925 22:05:17.089164 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:17.089172 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:17.089212 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:17.089766 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:17.089767 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:17.089769 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:17.089802 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:17.089804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.090956 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.090957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:17.092171 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:17.092178 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.092868 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.092869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:17.092930 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:17.092935 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:05:17.092963 (+ 28us) client_negotiation.cc:240] Negotiation successful
0925 22:05:17.093006 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":66,"threads_started":1}
I20260925 22:05:17.093508 7354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:17.088609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:37045 (local address 127.0.0.1:37280)
0925 22:05:17.088974 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:17.088986 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:17.089052 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:17.089466 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:17.089467 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:17.089469 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:17.089511 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:17.089513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.090776 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.090777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:17.092032 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:17.092034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.092971 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.092973 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:17.093037 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:17.093041 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:05:17.093378 (+ 337us) client_negotiation.cc:240] Negotiation successful
0925 22:05:17.093433 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":59,"thread_start_us":83,"threads_started":1}
I20260925 22:05:17.093659 6991 tablet_service.cc:1467] Processing CreateTablet for tablet 67d1e4bb1c134fea8befdd288a62a6f4 (DEFAULT_TABLE table=test-table [id=f9b04aa572ef40148138f5a1ee61e5e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:17.094023 6991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67d1e4bb1c134fea8befdd288a62a6f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:17.094147 7271 tablet_service.cc:1467] Processing CreateTablet for tablet 67d1e4bb1c134fea8befdd288a62a6f4 (DEFAULT_TABLE table=test-table [id=f9b04aa572ef40148138f5a1ee61e5e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:17.094450 7271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67d1e4bb1c134fea8befdd288a62a6f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:17.093726 7359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:17.089021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42444 (local address 127.5.0.66:33273)
0925 22:05:17.089600 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:17.089604 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:17.089623 (+ 19us) server_negotiation.cc:407] Connection header received
0925 22:05:17.089660 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:17.089661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:17.089675 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:17.089754 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:17.089864 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.090878 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.092253 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.092772 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.093452 (+ 680us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:17.093513 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:17.093537 (+ 24us) server_negotiation.cc:299] Negotiation successful
0925 22:05:17.093632 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":107,"threads_started":1}
I20260925 22:05:17.097573 7360 tablet_bootstrap.cc:492] T 67d1e4bb1c134fea8befdd288a62a6f4 P 38876e68cdfa46158375464729ee36f4: Bootstrap starting.
I20260925 22:05:17.097682 7361 tablet_bootstrap.cc:492] T 67d1e4bb1c134fea8befdd288a62a6f4 P 874ee8b794ef4086a9601b6a2df8dee6: Bootstrap starting.
I20260925 22:05:17.098184 7131 tablet_service.cc:1467] Processing CreateTablet for tablet 67d1e4bb1c134fea8befdd288a62a6f4 (DEFAULT_TABLE table=test-table [id=f9b04aa572ef40148138f5a1ee61e5e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:17.098388 7361 tablet_bootstrap.cc:654] T 67d1e4bb1c134fea8befdd288a62a6f4 P 874ee8b794ef4086a9601b6a2df8dee6: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:17.098485 7131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67d1e4bb1c134fea8befdd288a62a6f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:17.098645 7360 tablet_bootstrap.cc:654] T 67d1e4bb1c134fea8befdd288a62a6f4 P 38876e68cdfa46158375464729ee36f4: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:17.098743 7361 log.cc:824] T 67d1e4bb1c134fea8befdd288a62a6f4 P 874ee8b794ef4086a9601b6a2df8dee6: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:17.098965 7360 log.cc:824] T 67d1e4bb1c134fea8befdd288a62a6f4 P 38876e68cdfa46158375464729ee36f4: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:17.099507 7361 tablet_bootstrap.cc:492] T 67d1e4bb1c134fea8befdd288a62a6f4 P 874ee8b794ef4086a9601b6a2df8dee6: No bootstrap required, opened a new log
I20260925 22:05:17.099582 7361 ts_tablet_manager.cc:1397] T 67d1e4bb1c134fea8befdd288a62a6f4 P 874ee8b794ef4086a9601b6a2df8dee6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:05:17.099817 7360 tablet_bootstrap.cc:492] T 67d1e4bb1c134fea8befdd288a62a6f4 P 38876e68cdfa46158375464729ee36f4: No bootstrap required, opened a new log
I20260925 22:05:17.099911 7360 ts_tablet_manager.cc:1397] T 67d1e4bb1c134fea8befdd288a62a6f4 P 38876e68cdfa46158375464729ee36f4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:05:17.101214 7360 raft_consensus.cc:348] T 67d1e4bb1c134fea8befdd288a62a6f4 P 38876e68cdfa46158375464729ee36f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } }
I20260925 22:05:17.101365 7360 raft_consensus.cc:374] T 67d1e4bb1c134fea8befdd288a62a6f4 P 38876e68cdfa46158375464729ee36f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:17.101408 7360 raft_consensus.cc:729] T 67d1e4bb1c134fea8befdd288a62a6f4 P 38876e68cdfa46158375464729ee36f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38876e68cdfa46158375464729ee36f4, State: Initialized, Role: FOLLOWER
I20260925 22:05:17.101512 7360 consensus_queue.cc:260] T 67d1e4bb1c134fea8befdd288a62a6f4 P 38876e68cdfa46158375464729ee36f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } }
I20260925 22:05:17.101637 7361 raft_consensus.cc:348] T 67d1e4bb1c134fea8befdd288a62a6f4 P 874ee8b794ef4086a9601b6a2df8dee6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } }
I20260925 22:05:17.101802 7337 heartbeater.cc:503] Master 127.5.0.126:34405 was elected leader, sending a full tablet report...
I20260925 22:05:17.101804 7361 raft_consensus.cc:374] T 67d1e4bb1c134fea8befdd288a62a6f4 P 874ee8b794ef4086a9601b6a2df8dee6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:17.101848 7361 raft_consensus.cc:729] T 67d1e4bb1c134fea8befdd288a62a6f4 P 874ee8b794ef4086a9601b6a2df8dee6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 874ee8b794ef4086a9601b6a2df8dee6, State: Initialized, Role: FOLLOWER
I20260925 22:05:17.101989 7361 consensus_queue.cc:260] T 67d1e4bb1c134fea8befdd288a62a6f4 P 874ee8b794ef4086a9601b6a2df8dee6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } }
I20260925 22:05:17.102057 7360 ts_tablet_manager.cc:1428] T 67d1e4bb1c134fea8befdd288a62a6f4 P 38876e68cdfa46158375464729ee36f4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:05:17.102267 7361 ts_tablet_manager.cc:1428] T 67d1e4bb1c134fea8befdd288a62a6f4 P 874ee8b794ef4086a9601b6a2df8dee6: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260925 22:05:17.102398 7366 tablet_bootstrap.cc:492] T 67d1e4bb1c134fea8befdd288a62a6f4 P 10148bd29f0445239c5b4e9793900f5f: Bootstrap starting.
I20260925 22:05:17.102450 7057 heartbeater.cc:503] Master 127.5.0.126:34405 was elected leader, sending a full tablet report...
I20260925 22:05:17.103010 7366 tablet_bootstrap.cc:654] T 67d1e4bb1c134fea8befdd288a62a6f4 P 10148bd29f0445239c5b4e9793900f5f: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:17.103322 7366 log.cc:824] T 67d1e4bb1c134fea8befdd288a62a6f4 P 10148bd29f0445239c5b4e9793900f5f: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:17.104117 7366 tablet_bootstrap.cc:492] T 67d1e4bb1c134fea8befdd288a62a6f4 P 10148bd29f0445239c5b4e9793900f5f: No bootstrap required, opened a new log
I20260925 22:05:17.104189 7366 ts_tablet_manager.cc:1397] T 67d1e4bb1c134fea8befdd288a62a6f4 P 10148bd29f0445239c5b4e9793900f5f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:05:17.105460 7366 raft_consensus.cc:348] T 67d1e4bb1c134fea8befdd288a62a6f4 P 10148bd29f0445239c5b4e9793900f5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } }
I20260925 22:05:17.105600 7366 raft_consensus.cc:374] T 67d1e4bb1c134fea8befdd288a62a6f4 P 10148bd29f0445239c5b4e9793900f5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:17.105656 7366 raft_consensus.cc:729] T 67d1e4bb1c134fea8befdd288a62a6f4 P 10148bd29f0445239c5b4e9793900f5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10148bd29f0445239c5b4e9793900f5f, State: Initialized, Role: FOLLOWER
I20260925 22:05:17.105764 7366 consensus_queue.cc:260] T 67d1e4bb1c134fea8befdd288a62a6f4 P 10148bd29f0445239c5b4e9793900f5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } }
I20260925 22:05:17.106005 7366 ts_tablet_manager.cc:1428] T 67d1e4bb1c134fea8befdd288a62a6f4 P 10148bd29f0445239c5b4e9793900f5f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:05:17.106070 7197 heartbeater.cc:503] Master 127.5.0.126:34405 was elected leader, sending a full tablet report...
I20260925 22:05:17.132383 7365 raft_consensus.cc:482] T 67d1e4bb1c134fea8befdd288a62a6f4 P 874ee8b794ef4086a9601b6a2df8dee6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260925 22:05:17.132576 7198 tablet.cc:2369] T 67d1e4bb1c134fea8befdd288a62a6f4 P 10148bd29f0445239c5b4e9793900f5f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:05:17.132536 7365 raft_consensus.cc:504] T 67d1e4bb1c134fea8befdd288a62a6f4 P 874ee8b794ef4086a9601b6a2df8dee6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } }
I20260925 22:05:17.132912 7365 leader_election.cc:288] T 67d1e4bb1c134fea8befdd288a62a6f4 P 874ee8b794ef4086a9601b6a2df8dee6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38876e68cdfa46158375464729ee36f4 (127.5.0.67:37045), 10148bd29f0445239c5b4e9793900f5f (127.5.0.66:33273)
I20260925 22:05:17.135803 6950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:17.133100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:33273 (local address 127.5.0.65:49059)
0925 22:05:17.133157 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:17.133168 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:17.133209 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:17.133469 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:17.133471 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:17.133473 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:17.133548 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:17.133551 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.134229 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.134230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:17.135000 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:17.135002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.135544 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.135545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:17.135631 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:17.135635 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:05:17.135677 (+ 42us) client_negotiation.cc:240] Negotiation successful
0925 22:05:17.135709 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260925 22:05:17.135814 7359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:17.133299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:49059 (local address 127.5.0.66:33273)
0925 22:05:17.133342 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:17.133344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:17.133353 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:05:17.133378 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:17.133379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:17.133391 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:17.133446 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:17.133628 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.134144 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.135094 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.135462 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.135516 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:17.135524 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:17.135704 (+ 180us) server_negotiation.cc:299] Negotiation successful
0925 22:05:17.135749 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 22:05:17.135905 7356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:17.133265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:46617 (local address 127.5.0.67:37045)
0925 22:05:17.133310 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:17.133312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:17.133408 (+ 96us) server_negotiation.cc:407] Connection header received
0925 22:05:17.133455 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:17.133456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:17.133468 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:17.133530 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:17.133742 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.134225 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.135129 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.135476 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.135563 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:17.135572 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:17.135765 (+ 193us) server_negotiation.cc:299] Negotiation successful
0925 22:05:17.135805 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 22:05:17.135974 7369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:17.133100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:37045 (local address 127.5.0.65:46617)
0925 22:05:17.133328 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:17.133337 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:17.133395 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:17.133554 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:17.133555 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:17.133556 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:17.133666 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:17.133667 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.134309 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.134310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:17.135011 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:17.135013 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.135583 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.135584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:17.135681 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:17.135686 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:05:17.135744 (+ 58us) client_negotiation.cc:240] Negotiation successful
0925 22:05:17.135782 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":65,"thread_start_us":105,"threads_started":1}
I20260925 22:05:17.136180 7151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67d1e4bb1c134fea8befdd288a62a6f4" candidate_uuid: "874ee8b794ef4086a9601b6a2df8dee6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10148bd29f0445239c5b4e9793900f5f" is_pre_election: true
I20260925 22:05:17.136294 7151 raft_consensus.cc:2454] T 67d1e4bb1c134fea8befdd288a62a6f4 P 10148bd29f0445239c5b4e9793900f5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 874ee8b794ef4086a9601b6a2df8dee6 in term 0.
I20260925 22:05:17.136480 6942 leader_election.cc:302] T 67d1e4bb1c134fea8befdd288a62a6f4 P 874ee8b794ef4086a9601b6a2df8dee6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10148bd29f0445239c5b4e9793900f5f, 874ee8b794ef4086a9601b6a2df8dee6; no voters:
I20260925 22:05:17.136523 7291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67d1e4bb1c134fea8befdd288a62a6f4" candidate_uuid: "874ee8b794ef4086a9601b6a2df8dee6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38876e68cdfa46158375464729ee36f4" is_pre_election: true
I20260925 22:05:17.136608 7365 raft_consensus.cc:2781] T 67d1e4bb1c134fea8befdd288a62a6f4 P 874ee8b794ef4086a9601b6a2df8dee6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:05:17.136675 7291 raft_consensus.cc:2454] T 67d1e4bb1c134fea8befdd288a62a6f4 P 38876e68cdfa46158375464729ee36f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 874ee8b794ef4086a9601b6a2df8dee6 in term 0.
I20260925 22:05:17.136706 7365 raft_consensus.cc:482] T 67d1e4bb1c134fea8befdd288a62a6f4 P 874ee8b794ef4086a9601b6a2df8dee6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:05:17.136736 7365 raft_consensus.cc:3037] T 67d1e4bb1c134fea8befdd288a62a6f4 P 874ee8b794ef4086a9601b6a2df8dee6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:17.137449 7365 raft_consensus.cc:504] T 67d1e4bb1c134fea8befdd288a62a6f4 P 874ee8b794ef4086a9601b6a2df8dee6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } }
I20260925 22:05:17.137594 7365 leader_election.cc:288] T 67d1e4bb1c134fea8befdd288a62a6f4 P 874ee8b794ef4086a9601b6a2df8dee6 [CANDIDATE]: Term 1 election: Requested vote from peers 38876e68cdfa46158375464729ee36f4 (127.5.0.67:37045), 10148bd29f0445239c5b4e9793900f5f (127.5.0.66:33273)
I20260925 22:05:17.137737 7151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67d1e4bb1c134fea8befdd288a62a6f4" candidate_uuid: "874ee8b794ef4086a9601b6a2df8dee6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10148bd29f0445239c5b4e9793900f5f"
I20260925 22:05:17.137737 7291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67d1e4bb1c134fea8befdd288a62a6f4" candidate_uuid: "874ee8b794ef4086a9601b6a2df8dee6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38876e68cdfa46158375464729ee36f4"
I20260925 22:05:17.137810 7291 raft_consensus.cc:3037] T 67d1e4bb1c134fea8befdd288a62a6f4 P 38876e68cdfa46158375464729ee36f4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:17.137810 7151 raft_consensus.cc:3037] T 67d1e4bb1c134fea8befdd288a62a6f4 P 10148bd29f0445239c5b4e9793900f5f [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:17.138635 7291 raft_consensus.cc:2454] T 67d1e4bb1c134fea8befdd288a62a6f4 P 38876e68cdfa46158375464729ee36f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 874ee8b794ef4086a9601b6a2df8dee6 in term 1.
I20260925 22:05:17.138635 7151 raft_consensus.cc:2454] T 67d1e4bb1c134fea8befdd288a62a6f4 P 10148bd29f0445239c5b4e9793900f5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 874ee8b794ef4086a9601b6a2df8dee6 in term 1.
I20260925 22:05:17.138814 6942 leader_election.cc:302] T 67d1e4bb1c134fea8befdd288a62a6f4 P 874ee8b794ef4086a9601b6a2df8dee6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10148bd29f0445239c5b4e9793900f5f, 874ee8b794ef4086a9601b6a2df8dee6; no voters:
I20260925 22:05:17.138964 7365 raft_consensus.cc:2781] T 67d1e4bb1c134fea8befdd288a62a6f4 P 874ee8b794ef4086a9601b6a2df8dee6 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:17.139101 7365 raft_consensus.cc:686] T 67d1e4bb1c134fea8befdd288a62a6f4 P 874ee8b794ef4086a9601b6a2df8dee6 [term 1 LEADER]: Becoming Leader. State: Replica: 874ee8b794ef4086a9601b6a2df8dee6, State: Running, Role: LEADER
I20260925 22:05:17.139220 7365 consensus_queue.cc:237] T 67d1e4bb1c134fea8befdd288a62a6f4 P 874ee8b794ef4086a9601b6a2df8dee6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } }
I20260925 22:05:17.140026 6854 catalog_manager.cc:5495] T 67d1e4bb1c134fea8befdd288a62a6f4 P 874ee8b794ef4086a9601b6a2df8dee6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 874ee8b794ef4086a9601b6a2df8dee6 (127.5.0.65). New cstate: current_term: 1 leader_uuid: "874ee8b794ef4086a9601b6a2df8dee6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } health_report { overall_health: UNKNOWN } } }
Sep 25 22:05:17 dist-test-slave-7mqr krb5kdc[6809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373916, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:17.175621 7377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:17.168654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34405 (local address 127.0.0.1:50430)
0925 22:05:17.168829 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:17.168848 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:17.168913 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:17.169052 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:17.169053 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:17.169374 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:17.169421 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:17.169422 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.170040 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.170041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:17.170561 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:17.170562 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.170873 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.170874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:17.170913 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:17.171445 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:17.171450 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:17.172961 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:17.174542 (+ 1581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:17.174552 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:17.174555 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:17.174761 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:17.174952 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:17.174953 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:17.174953 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:17.175015 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:17.175306 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:17.175312 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:17.175356 (+ 44us) client_negotiation.cc:769] Sending connection context
0925 22:05:17.175412 (+ 56us) client_negotiation.cc:240] Negotiation successful
0925 22:05:17.175523 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":3,"thread_start_us":106,"threads_started":1}
I20260925 22:05:17.175671 6903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:17.168694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50430 (local address 127.5.0.126:34405)
0925 22:05:17.168770 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:17.168772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:17.168916 (+ 144us) server_negotiation.cc:407] Connection header received
0925 22:05:17.168960 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:17.168960 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:17.168969 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:17.169032 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:17.169491 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.169973 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.170638 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.170810 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.170861 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:17.173048 (+ 2187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:17.173050 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:17.173050 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:17.173057 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:17.174458 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:17.174828 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:17.174829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:17.174829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:17.174878 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:17.175077 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:17.175078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:17.175078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:17.175242 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:17.175299 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:17.175435 (+ 136us) server_negotiation.cc:299] Negotiation successful
0925 22:05:17.175567 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":50}
I20260925 22:05:17.177639 6854 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:50430:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 22:05:17.177816 6854 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:05:17.179297 7131 tablet_service.cc:1467] Processing CreateTablet for tablet b3b299675c1a45cf8ab40e5d98119a0d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4934aa4fedcf4b9a9aad1ebaaa9be337]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 22:05:17.179347 7271 tablet_service.cc:1467] Processing CreateTablet for tablet b3b299675c1a45cf8ab40e5d98119a0d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4934aa4fedcf4b9a9aad1ebaaa9be337]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 22:05:17.179379 6991 tablet_service.cc:1467] Processing CreateTablet for tablet b3b299675c1a45cf8ab40e5d98119a0d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4934aa4fedcf4b9a9aad1ebaaa9be337]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 22:05:17.179481 7131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3b299675c1a45cf8ab40e5d98119a0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:17.179481 7271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3b299675c1a45cf8ab40e5d98119a0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:17.179559 6991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3b299675c1a45cf8ab40e5d98119a0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:17.180804 7361 tablet_bootstrap.cc:492] T b3b299675c1a45cf8ab40e5d98119a0d P 874ee8b794ef4086a9601b6a2df8dee6: Bootstrap starting.
I20260925 22:05:17.181017 7360 tablet_bootstrap.cc:492] T b3b299675c1a45cf8ab40e5d98119a0d P 38876e68cdfa46158375464729ee36f4: Bootstrap starting.
I20260925 22:05:17.181025 7366 tablet_bootstrap.cc:492] T b3b299675c1a45cf8ab40e5d98119a0d P 10148bd29f0445239c5b4e9793900f5f: Bootstrap starting.
I20260925 22:05:17.181283 7361 tablet_bootstrap.cc:654] T b3b299675c1a45cf8ab40e5d98119a0d P 874ee8b794ef4086a9601b6a2df8dee6: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:17.181571 7360 tablet_bootstrap.cc:654] T b3b299675c1a45cf8ab40e5d98119a0d P 38876e68cdfa46158375464729ee36f4: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:17.181732 7366 tablet_bootstrap.cc:654] T b3b299675c1a45cf8ab40e5d98119a0d P 10148bd29f0445239c5b4e9793900f5f: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:17.182076 7361 tablet_bootstrap.cc:492] T b3b299675c1a45cf8ab40e5d98119a0d P 874ee8b794ef4086a9601b6a2df8dee6: No bootstrap required, opened a new log
I20260925 22:05:17.182127 7361 ts_tablet_manager.cc:1397] T b3b299675c1a45cf8ab40e5d98119a0d P 874ee8b794ef4086a9601b6a2df8dee6: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 22:05:17.182279 7361 raft_consensus.cc:348] T b3b299675c1a45cf8ab40e5d98119a0d P 874ee8b794ef4086a9601b6a2df8dee6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:17.182364 7361 raft_consensus.cc:374] T b3b299675c1a45cf8ab40e5d98119a0d P 874ee8b794ef4086a9601b6a2df8dee6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:17.182389 7361 raft_consensus.cc:729] T b3b299675c1a45cf8ab40e5d98119a0d P 874ee8b794ef4086a9601b6a2df8dee6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 874ee8b794ef4086a9601b6a2df8dee6, State: Initialized, Role: FOLLOWER
I20260925 22:05:17.182440 7361 consensus_queue.cc:260] T b3b299675c1a45cf8ab40e5d98119a0d P 874ee8b794ef4086a9601b6a2df8dee6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:17.182540 7361 ts_tablet_manager.cc:1428] T b3b299675c1a45cf8ab40e5d98119a0d P 874ee8b794ef4086a9601b6a2df8dee6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:17.182614 7365 tablet_replica.cc:439] T b3b299675c1a45cf8ab40e5d98119a0d P 874ee8b794ef4086a9601b6a2df8dee6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } } }
I20260925 22:05:17.182730 7365 tablet_replica.cc:442] T b3b299675c1a45cf8ab40e5d98119a0d P 874ee8b794ef4086a9601b6a2df8dee6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:05:17.183121 7366 tablet_bootstrap.cc:492] T b3b299675c1a45cf8ab40e5d98119a0d P 10148bd29f0445239c5b4e9793900f5f: No bootstrap required, opened a new log
I20260925 22:05:17.183183 7366 ts_tablet_manager.cc:1397] T b3b299675c1a45cf8ab40e5d98119a0d P 10148bd29f0445239c5b4e9793900f5f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:05:17.183182 7360 tablet_bootstrap.cc:492] T b3b299675c1a45cf8ab40e5d98119a0d P 38876e68cdfa46158375464729ee36f4: No bootstrap required, opened a new log
I20260925 22:05:17.183241 7360 ts_tablet_manager.cc:1397] T b3b299675c1a45cf8ab40e5d98119a0d P 38876e68cdfa46158375464729ee36f4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:05:17.183321 7366 raft_consensus.cc:348] T b3b299675c1a45cf8ab40e5d98119a0d P 10148bd29f0445239c5b4e9793900f5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:17.183369 7366 raft_consensus.cc:374] T b3b299675c1a45cf8ab40e5d98119a0d P 10148bd29f0445239c5b4e9793900f5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:17.183382 7366 raft_consensus.cc:729] T b3b299675c1a45cf8ab40e5d98119a0d P 10148bd29f0445239c5b4e9793900f5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10148bd29f0445239c5b4e9793900f5f, State: Initialized, Role: FOLLOWER
I20260925 22:05:17.183369 7360 raft_consensus.cc:348] T b3b299675c1a45cf8ab40e5d98119a0d P 38876e68cdfa46158375464729ee36f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:17.183418 7360 raft_consensus.cc:374] T b3b299675c1a45cf8ab40e5d98119a0d P 38876e68cdfa46158375464729ee36f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:17.183430 7360 raft_consensus.cc:729] T b3b299675c1a45cf8ab40e5d98119a0d P 38876e68cdfa46158375464729ee36f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38876e68cdfa46158375464729ee36f4, State: Initialized, Role: FOLLOWER
I20260925 22:05:17.183440 7366 consensus_queue.cc:260] T b3b299675c1a45cf8ab40e5d98119a0d P 10148bd29f0445239c5b4e9793900f5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:17.183472 7360 consensus_queue.cc:260] T b3b299675c1a45cf8ab40e5d98119a0d P 38876e68cdfa46158375464729ee36f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:17.183526 7366 ts_tablet_manager.cc:1428] T b3b299675c1a45cf8ab40e5d98119a0d P 10148bd29f0445239c5b4e9793900f5f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:17.183565 7360 ts_tablet_manager.cc:1428] T b3b299675c1a45cf8ab40e5d98119a0d P 38876e68cdfa46158375464729ee36f4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:17.183624 7368 tablet_replica.cc:439] T b3b299675c1a45cf8ab40e5d98119a0d P 10148bd29f0445239c5b4e9793900f5f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } } }
I20260925 22:05:17.183653 7364 tablet_replica.cc:439] T b3b299675c1a45cf8ab40e5d98119a0d P 38876e68cdfa46158375464729ee36f4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } } }
I20260925 22:05:17.183706 7368 tablet_replica.cc:442] T b3b299675c1a45cf8ab40e5d98119a0d P 10148bd29f0445239c5b4e9793900f5f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:05:17.183738 7364 tablet_replica.cc:442] T b3b299675c1a45cf8ab40e5d98119a0d P 38876e68cdfa46158375464729ee36f4: This TxnStatusTablet replica's current role is: FOLLOWER
W20260925 22:05:17.214525 7058 tablet.cc:2369] T 67d1e4bb1c134fea8befdd288a62a6f4 P 874ee8b794ef4086a9601b6a2df8dee6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:05:17.237918 7365 raft_consensus.cc:482] T b3b299675c1a45cf8ab40e5d98119a0d P 874ee8b794ef4086a9601b6a2df8dee6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:05:17.238013 7365 raft_consensus.cc:504] T b3b299675c1a45cf8ab40e5d98119a0d P 874ee8b794ef4086a9601b6a2df8dee6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:17.238183 7365 leader_election.cc:288] T b3b299675c1a45cf8ab40e5d98119a0d P 874ee8b794ef4086a9601b6a2df8dee6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10148bd29f0445239c5b4e9793900f5f (127.5.0.66:33273), 38876e68cdfa46158375464729ee36f4 (127.5.0.67:37045)
I20260925 22:05:17.238411 7151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b299675c1a45cf8ab40e5d98119a0d" candidate_uuid: "874ee8b794ef4086a9601b6a2df8dee6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10148bd29f0445239c5b4e9793900f5f" is_pre_election: true
I20260925 22:05:17.238411 7291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b299675c1a45cf8ab40e5d98119a0d" candidate_uuid: "874ee8b794ef4086a9601b6a2df8dee6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38876e68cdfa46158375464729ee36f4" is_pre_election: true
I20260925 22:05:17.238507 7151 raft_consensus.cc:2454] T b3b299675c1a45cf8ab40e5d98119a0d P 10148bd29f0445239c5b4e9793900f5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 874ee8b794ef4086a9601b6a2df8dee6 in term 0.
I20260925 22:05:17.238507 7291 raft_consensus.cc:2454] T b3b299675c1a45cf8ab40e5d98119a0d P 38876e68cdfa46158375464729ee36f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 874ee8b794ef4086a9601b6a2df8dee6 in term 0.
I20260925 22:05:17.238655 6942 leader_election.cc:302] T b3b299675c1a45cf8ab40e5d98119a0d P 874ee8b794ef4086a9601b6a2df8dee6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10148bd29f0445239c5b4e9793900f5f, 874ee8b794ef4086a9601b6a2df8dee6; no voters:
I20260925 22:05:17.238786 7365 raft_consensus.cc:2781] T b3b299675c1a45cf8ab40e5d98119a0d P 874ee8b794ef4086a9601b6a2df8dee6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:05:17.238830 7365 raft_consensus.cc:482] T b3b299675c1a45cf8ab40e5d98119a0d P 874ee8b794ef4086a9601b6a2df8dee6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:05:17.238862 7365 raft_consensus.cc:3037] T b3b299675c1a45cf8ab40e5d98119a0d P 874ee8b794ef4086a9601b6a2df8dee6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:17.239596 7365 raft_consensus.cc:504] T b3b299675c1a45cf8ab40e5d98119a0d P 874ee8b794ef4086a9601b6a2df8dee6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:17.239761 7365 leader_election.cc:288] T b3b299675c1a45cf8ab40e5d98119a0d P 874ee8b794ef4086a9601b6a2df8dee6 [CANDIDATE]: Term 1 election: Requested vote from peers 10148bd29f0445239c5b4e9793900f5f (127.5.0.66:33273), 38876e68cdfa46158375464729ee36f4 (127.5.0.67:37045)
I20260925 22:05:17.239964 7151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b299675c1a45cf8ab40e5d98119a0d" candidate_uuid: "874ee8b794ef4086a9601b6a2df8dee6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10148bd29f0445239c5b4e9793900f5f"
I20260925 22:05:17.239995 7291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b299675c1a45cf8ab40e5d98119a0d" candidate_uuid: "874ee8b794ef4086a9601b6a2df8dee6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38876e68cdfa46158375464729ee36f4"
I20260925 22:05:17.240058 7291 raft_consensus.cc:3037] T b3b299675c1a45cf8ab40e5d98119a0d P 38876e68cdfa46158375464729ee36f4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:17.240062 7151 raft_consensus.cc:3037] T b3b299675c1a45cf8ab40e5d98119a0d P 10148bd29f0445239c5b4e9793900f5f [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:17.240784 7291 raft_consensus.cc:2454] T b3b299675c1a45cf8ab40e5d98119a0d P 38876e68cdfa46158375464729ee36f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 874ee8b794ef4086a9601b6a2df8dee6 in term 1.
I20260925 22:05:17.240808 7151 raft_consensus.cc:2454] T b3b299675c1a45cf8ab40e5d98119a0d P 10148bd29f0445239c5b4e9793900f5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 874ee8b794ef4086a9601b6a2df8dee6 in term 1.
I20260925 22:05:17.240974 6943 leader_election.cc:302] T b3b299675c1a45cf8ab40e5d98119a0d P 874ee8b794ef4086a9601b6a2df8dee6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38876e68cdfa46158375464729ee36f4, 874ee8b794ef4086a9601b6a2df8dee6; no voters:
I20260925 22:05:17.241145 7365 raft_consensus.cc:2781] T b3b299675c1a45cf8ab40e5d98119a0d P 874ee8b794ef4086a9601b6a2df8dee6 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:17.241204 7365 raft_consensus.cc:686] T b3b299675c1a45cf8ab40e5d98119a0d P 874ee8b794ef4086a9601b6a2df8dee6 [term 1 LEADER]: Becoming Leader. State: Replica: 874ee8b794ef4086a9601b6a2df8dee6, State: Running, Role: LEADER
I20260925 22:05:17.241266 7365 consensus_queue.cc:237] T b3b299675c1a45cf8ab40e5d98119a0d P 874ee8b794ef4086a9601b6a2df8dee6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:17.241560 7370 tablet_replica.cc:439] T b3b299675c1a45cf8ab40e5d98119a0d P 874ee8b794ef4086a9601b6a2df8dee6: TxnStatusTablet state changed. Reason: New leader 874ee8b794ef4086a9601b6a2df8dee6. Latest consensus state: current_term: 1 leader_uuid: "874ee8b794ef4086a9601b6a2df8dee6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } } }
I20260925 22:05:17.241643 7370 tablet_replica.cc:442] T b3b299675c1a45cf8ab40e5d98119a0d P 874ee8b794ef4086a9601b6a2df8dee6: This TxnStatusTablet replica's current role is: LEADER
I20260925 22:05:17.241863 6853 catalog_manager.cc:5495] T b3b299675c1a45cf8ab40e5d98119a0d P 874ee8b794ef4086a9601b6a2df8dee6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 874ee8b794ef4086a9601b6a2df8dee6 (127.5.0.65). New cstate: current_term: 1 leader_uuid: "874ee8b794ef4086a9601b6a2df8dee6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } health_report { overall_health: UNKNOWN } } }
I20260925 22:05:17.242179 7382 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 22:05:17.242327 7382 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 22:05:17.258445 7357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:17.255964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44026 (local address 127.5.0.65:36769)
0925 22:05:17.256012 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:17.256014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:17.256025 (+ 11us) server_negotiation.cc:407] Connection header received
0925 22:05:17.256116 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:17.256117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:17.256142 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:17.256211 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:05:17.256367 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.256845 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.257600 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.257802 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.257888 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:17.258085 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:05:17.258187 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:05:17.258282 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:17.258324 (+ 42us) server_negotiation.cc:299] Negotiation successful
0925 22:05:17.258343 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 22:05:17.258450 7377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:17.255881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:36769 (local address 127.0.0.1:44026)
0925 22:05:17.255968 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:17.255981 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:17.256033 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:17.256236 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:17.256237 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:17.256239 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 22:05:17.256289 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:17.256292 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.256931 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.256932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:17.257513 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:17.257514 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.257888 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.257889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:17.257957 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:17.257975 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 22:05:17.258260 (+ 285us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 22:05:17.258269 (+ 9us) client_negotiation.cc:769] Sending connection context
0925 22:05:17.258318 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 22:05:17.258373 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260925 22:05:17.260349 7151 raft_consensus.cc:1261] T b3b299675c1a45cf8ab40e5d98119a0d P 10148bd29f0445239c5b4e9793900f5f [term 1 FOLLOWER]: Refusing update from remote peer 874ee8b794ef4086a9601b6a2df8dee6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:05:17.260344 7291 raft_consensus.cc:1261] T b3b299675c1a45cf8ab40e5d98119a0d P 38876e68cdfa46158375464729ee36f4 [term 1 FOLLOWER]: Refusing update from remote peer 874ee8b794ef4086a9601b6a2df8dee6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:05:17.260756 7370 consensus_queue.cc:1035] T b3b299675c1a45cf8ab40e5d98119a0d P 874ee8b794ef4086a9601b6a2df8dee6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:05:17.260888 7365 consensus_queue.cc:1035] T b3b299675c1a45cf8ab40e5d98119a0d P 874ee8b794ef4086a9601b6a2df8dee6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:05:17.262058 7368 tablet_replica.cc:439] T b3b299675c1a45cf8ab40e5d98119a0d P 10148bd29f0445239c5b4e9793900f5f: TxnStatusTablet state changed. Reason: New leader 874ee8b794ef4086a9601b6a2df8dee6. Latest consensus state: current_term: 1 leader_uuid: "874ee8b794ef4086a9601b6a2df8dee6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } } }
I20260925 22:05:17.262063 7364 tablet_replica.cc:439] T b3b299675c1a45cf8ab40e5d98119a0d P 38876e68cdfa46158375464729ee36f4: TxnStatusTablet state changed. Reason: New leader 874ee8b794ef4086a9601b6a2df8dee6. Latest consensus state: current_term: 1 leader_uuid: "874ee8b794ef4086a9601b6a2df8dee6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } } }
I20260925 22:05:17.262168 7364 tablet_replica.cc:442] T b3b299675c1a45cf8ab40e5d98119a0d P 38876e68cdfa46158375464729ee36f4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:05:17.262169 7368 tablet_replica.cc:442] T b3b299675c1a45cf8ab40e5d98119a0d P 10148bd29f0445239c5b4e9793900f5f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:05:17.263489 7365 tablet_replica.cc:439] T b3b299675c1a45cf8ab40e5d98119a0d P 874ee8b794ef4086a9601b6a2df8dee6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "874ee8b794ef4086a9601b6a2df8dee6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } } }
I20260925 22:05:17.263576 7365 tablet_replica.cc:442] T b3b299675c1a45cf8ab40e5d98119a0d P 874ee8b794ef4086a9601b6a2df8dee6: This TxnStatusTablet replica's current role is: LEADER
I20260925 22:05:17.264050 7385 mvcc.cc:205] Tried to move back new op lower bound from 7333371565095657472 to 7333371565020700672. Current Snapshot: MvccSnapshot[applied={T|T < 7333371565095657472}]
I20260925 22:05:17.264164 7365 tablet_replica.cc:439] T b3b299675c1a45cf8ab40e5d98119a0d P 874ee8b794ef4086a9601b6a2df8dee6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "874ee8b794ef4086a9601b6a2df8dee6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } } }
I20260925 22:05:17.264238 7365 tablet_replica.cc:442] T b3b299675c1a45cf8ab40e5d98119a0d P 874ee8b794ef4086a9601b6a2df8dee6: This TxnStatusTablet replica's current role is: LEADER
I20260925 22:05:17.264351 7387 mvcc.cc:205] Tried to move back new op lower bound from 7333371565095657472 to 7333371565020700672. Current Snapshot: MvccSnapshot[applied={T|T < 7333371565095657472}]
I20260925 22:05:17.264451 7364 tablet_replica.cc:439] T b3b299675c1a45cf8ab40e5d98119a0d P 38876e68cdfa46158375464729ee36f4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "874ee8b794ef4086a9601b6a2df8dee6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } } }
I20260925 22:05:17.264523 7364 tablet_replica.cc:442] T b3b299675c1a45cf8ab40e5d98119a0d P 38876e68cdfa46158375464729ee36f4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:05:17.264843 7368 tablet_replica.cc:439] T b3b299675c1a45cf8ab40e5d98119a0d P 10148bd29f0445239c5b4e9793900f5f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "874ee8b794ef4086a9601b6a2df8dee6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } } }
I20260925 22:05:17.264943 7368 tablet_replica.cc:442] T b3b299675c1a45cf8ab40e5d98119a0d P 10148bd29f0445239c5b4e9793900f5f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:05:17.268983 7357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:17.266660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44028 (local address 127.5.0.65:36769)
0925 22:05:17.266703 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:17.266705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:17.266714 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:05:17.266777 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:17.266778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:17.266787 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:17.266831 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:05:17.266962 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.267424 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.268283 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.268490 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.268566 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:17.268733 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:05:17.268770 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:05:17.268829 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:17.268891 (+ 62us) server_negotiation.cc:299] Negotiation successful
0925 22:05:17.268909 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 22:05:17.274036 7357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:17.271325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:40593 (local address 127.5.0.65:36769)
0925 22:05:17.271361 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:17.271363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:17.271525 (+ 162us) server_negotiation.cc:407] Connection header received
0925 22:05:17.271580 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:17.271581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:17.271597 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:17.271642 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:05:17.271776 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.272260 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.273378 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.273548 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.273594 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:17.273801 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:05:17.273823 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:05:17.273860 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:17.273932 (+ 72us) server_negotiation.cc:299] Negotiation successful
0925 22:05:17.273950 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":17}
I20260925 22:05:17.274372 7396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:17.271292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:36769 (local address 127.5.0.65:40593)
0925 22:05:17.271452 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:17.271464 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:17.271525 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:17.271658 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:17.271659 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:17.271660 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 22:05:17.271719 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:17.271720 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.272327 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.272327 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:17.273289 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:17.273291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.273620 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.273621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:17.273694 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:17.273709 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 22:05:17.273877 (+ 168us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 22:05:17.273879 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 22:05:17.273928 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 22:05:17.273974 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":83,"threads_started":1}
I20260925 22:05:17.276178 7291 raft_consensus.cc:1261] T 67d1e4bb1c134fea8befdd288a62a6f4 P 38876e68cdfa46158375464729ee36f4 [term 1 FOLLOWER]: Refusing update from remote peer 874ee8b794ef4086a9601b6a2df8dee6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:05:17.276188 7150 raft_consensus.cc:1261] T 67d1e4bb1c134fea8befdd288a62a6f4 P 10148bd29f0445239c5b4e9793900f5f [term 1 FOLLOWER]: Refusing update from remote peer 874ee8b794ef4086a9601b6a2df8dee6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:05:17.276386 7365 consensus_queue.cc:1035] T 67d1e4bb1c134fea8befdd288a62a6f4 P 874ee8b794ef4086a9601b6a2df8dee6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:05:17.276473 7370 consensus_queue.cc:1035] T 67d1e4bb1c134fea8befdd288a62a6f4 P 874ee8b794ef4086a9601b6a2df8dee6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:05:17.277781 7386 mvcc.cc:205] Tried to move back new op lower bound from 7333371565161807872 to 7333371564602699776. Current Snapshot: MvccSnapshot[applied={T|T < 7333371565161807872}]
I20260925 22:05:17.289189 6853 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:05:17.289309 6853 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:05:17.290057 6853 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 196ba701dd2e4bb9a12bc02d704b73ce: Sending DeleteTablet for 3 replicas of tablet 67d1e4bb1c134fea8befdd288a62a6f4
I20260925 22:05:17.290381 6991 tablet_service.cc:1514] Processing DeleteTablet for tablet 67d1e4bb1c134fea8befdd288a62a6f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:17 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:44022
I20260925 22:05:17.290413 7131 tablet_service.cc:1514] Processing DeleteTablet for tablet 67d1e4bb1c134fea8befdd288a62a6f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:17 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:42444
I20260925 22:05:17.290445 7271 tablet_service.cc:1514] Processing DeleteTablet for tablet 67d1e4bb1c134fea8befdd288a62a6f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:17 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:37280
I20260925 22:05:17.290724 7404 tablet_replica.cc:330] T 67d1e4bb1c134fea8befdd288a62a6f4 P 38876e68cdfa46158375464729ee36f4: stopping tablet replica
I20260925 22:05:17.290834 7404 raft_consensus.cc:2229] T 67d1e4bb1c134fea8befdd288a62a6f4 P 38876e68cdfa46158375464729ee36f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:05:17.290892 7404 raft_consensus.cc:2258] T 67d1e4bb1c134fea8befdd288a62a6f4 P 38876e68cdfa46158375464729ee36f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:17.290940 6853 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50422:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:05:17.291098 6853 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:05:17.291308 7404 ts_tablet_manager.cc:1905] T 67d1e4bb1c134fea8befdd288a62a6f4 P 38876e68cdfa46158375464729ee36f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:17.292236 7271 tablet_service.cc:1467] Processing CreateTablet for tablet 27ecb5c0c3ba4af1b9493c20bd246145 (DEFAULT_TABLE table=test-table [id=7ca49adde8e84c03bb9d1e195c6028bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:17.292241 7131 tablet_service.cc:1467] Processing CreateTablet for tablet 27ecb5c0c3ba4af1b9493c20bd246145 (DEFAULT_TABLE table=test-table [id=7ca49adde8e84c03bb9d1e195c6028bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:17.292352 7131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27ecb5c0c3ba4af1b9493c20bd246145. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:17.292375 7271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27ecb5c0c3ba4af1b9493c20bd246145. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:17.293220 7404 ts_tablet_manager.cc:1918] T 67d1e4bb1c134fea8befdd288a62a6f4 P 38876e68cdfa46158375464729ee36f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 22:05:17.293289 7404 log.cc:1196] T 67d1e4bb1c134fea8befdd288a62a6f4 P 38876e68cdfa46158375464729ee36f4: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-2/wal/wals/67d1e4bb1c134fea8befdd288a62a6f4
I20260925 22:05:17.293561 7404 ts_tablet_manager.cc:1939] T 67d1e4bb1c134fea8befdd288a62a6f4 P 38876e68cdfa46158375464729ee36f4: Deleting consensus metadata
I20260925 22:05:17.293736 7366 tablet_bootstrap.cc:492] T 27ecb5c0c3ba4af1b9493c20bd246145 P 10148bd29f0445239c5b4e9793900f5f: Bootstrap starting.
I20260925 22:05:17.293843 7360 tablet_bootstrap.cc:492] T 27ecb5c0c3ba4af1b9493c20bd246145 P 38876e68cdfa46158375464729ee36f4: Bootstrap starting.
I20260925 22:05:17.294144 7366 tablet_bootstrap.cc:654] T 27ecb5c0c3ba4af1b9493c20bd246145 P 10148bd29f0445239c5b4e9793900f5f: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:17.294210 7360 tablet_bootstrap.cc:654] T 27ecb5c0c3ba4af1b9493c20bd246145 P 38876e68cdfa46158375464729ee36f4: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:17.294696 6991 tablet_service.cc:1467] Processing CreateTablet for tablet 27ecb5c0c3ba4af1b9493c20bd246145 (DEFAULT_TABLE table=test-table [id=7ca49adde8e84c03bb9d1e195c6028bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:17.294807 6991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27ecb5c0c3ba4af1b9493c20bd246145. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:17.294781 6839 catalog_manager.cc:4841] TS 38876e68cdfa46158375464729ee36f4 (127.5.0.67:37045): tablet 67d1e4bb1c134fea8befdd288a62a6f4 (table test-table [id=f9b04aa572ef40148138f5a1ee61e5e7]) successfully deleted
I20260925 22:05:17.294819 7360 tablet_bootstrap.cc:492] T 27ecb5c0c3ba4af1b9493c20bd246145 P 38876e68cdfa46158375464729ee36f4: No bootstrap required, opened a new log
I20260925 22:05:17.294859 7360 ts_tablet_manager.cc:1397] T 27ecb5c0c3ba4af1b9493c20bd246145 P 38876e68cdfa46158375464729ee36f4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 22:05:17.294991 7360 raft_consensus.cc:348] T 27ecb5c0c3ba4af1b9493c20bd246145 P 38876e68cdfa46158375464729ee36f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:17.295054 7360 raft_consensus.cc:374] T 27ecb5c0c3ba4af1b9493c20bd246145 P 38876e68cdfa46158375464729ee36f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:17.295074 7360 raft_consensus.cc:729] T 27ecb5c0c3ba4af1b9493c20bd246145 P 38876e68cdfa46158375464729ee36f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38876e68cdfa46158375464729ee36f4, State: Initialized, Role: FOLLOWER
I20260925 22:05:17.295113 7360 consensus_queue.cc:260] T 27ecb5c0c3ba4af1b9493c20bd246145 P 38876e68cdfa46158375464729ee36f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:17.295192 7360 ts_tablet_manager.cc:1428] T 27ecb5c0c3ba4af1b9493c20bd246145 P 38876e68cdfa46158375464729ee36f4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:17.295740 7366 tablet_bootstrap.cc:492] T 27ecb5c0c3ba4af1b9493c20bd246145 P 10148bd29f0445239c5b4e9793900f5f: No bootstrap required, opened a new log
I20260925 22:05:17.295799 7366 ts_tablet_manager.cc:1397] T 27ecb5c0c3ba4af1b9493c20bd246145 P 10148bd29f0445239c5b4e9793900f5f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:05:17.295946 7366 raft_consensus.cc:348] T 27ecb5c0c3ba4af1b9493c20bd246145 P 10148bd29f0445239c5b4e9793900f5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:17.296015 7366 raft_consensus.cc:374] T 27ecb5c0c3ba4af1b9493c20bd246145 P 10148bd29f0445239c5b4e9793900f5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:17.296036 7366 raft_consensus.cc:729] T 27ecb5c0c3ba4af1b9493c20bd246145 P 10148bd29f0445239c5b4e9793900f5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10148bd29f0445239c5b4e9793900f5f, State: Initialized, Role: FOLLOWER
I20260925 22:05:17.296074 7366 consensus_queue.cc:260] T 27ecb5c0c3ba4af1b9493c20bd246145 P 10148bd29f0445239c5b4e9793900f5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:17.296173 7366 ts_tablet_manager.cc:1428] T 27ecb5c0c3ba4af1b9493c20bd246145 P 10148bd29f0445239c5b4e9793900f5f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 22:05:17.296222 7361 tablet_bootstrap.cc:492] T 27ecb5c0c3ba4af1b9493c20bd246145 P 874ee8b794ef4086a9601b6a2df8dee6: Bootstrap starting.
I20260925 22:05:17.296612 7361 tablet_bootstrap.cc:654] T 27ecb5c0c3ba4af1b9493c20bd246145 P 874ee8b794ef4086a9601b6a2df8dee6: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:17.297235 7361 tablet_bootstrap.cc:492] T 27ecb5c0c3ba4af1b9493c20bd246145 P 874ee8b794ef4086a9601b6a2df8dee6: No bootstrap required, opened a new log
I20260925 22:05:17.297274 7361 ts_tablet_manager.cc:1397] T 27ecb5c0c3ba4af1b9493c20bd246145 P 874ee8b794ef4086a9601b6a2df8dee6: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 22:05:17.297394 7361 raft_consensus.cc:348] T 27ecb5c0c3ba4af1b9493c20bd246145 P 874ee8b794ef4086a9601b6a2df8dee6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:17.297461 7361 raft_consensus.cc:374] T 27ecb5c0c3ba4af1b9493c20bd246145 P 874ee8b794ef4086a9601b6a2df8dee6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:17.297482 7361 raft_consensus.cc:729] T 27ecb5c0c3ba4af1b9493c20bd246145 P 874ee8b794ef4086a9601b6a2df8dee6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 874ee8b794ef4086a9601b6a2df8dee6, State: Initialized, Role: FOLLOWER
I20260925 22:05:17.297523 7361 consensus_queue.cc:260] T 27ecb5c0c3ba4af1b9493c20bd246145 P 874ee8b794ef4086a9601b6a2df8dee6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:17.297605 7361 ts_tablet_manager.cc:1428] T 27ecb5c0c3ba4af1b9493c20bd246145 P 874ee8b794ef4086a9601b6a2df8dee6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:17.299994 7402 tablet_replica.cc:330] T 67d1e4bb1c134fea8befdd288a62a6f4 P 874ee8b794ef4086a9601b6a2df8dee6: stopping tablet replica
I20260925 22:05:17.300149 7402 raft_consensus.cc:2229] T 67d1e4bb1c134fea8befdd288a62a6f4 P 874ee8b794ef4086a9601b6a2df8dee6 [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:05:17.300299 7402 raft_consensus.cc:2258] T 67d1e4bb1c134fea8befdd288a62a6f4 P 874ee8b794ef4086a9601b6a2df8dee6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:17.300510 7403 tablet_replica.cc:330] T 67d1e4bb1c134fea8befdd288a62a6f4 P 10148bd29f0445239c5b4e9793900f5f: stopping tablet replica
I20260925 22:05:17.300993 7402 ts_tablet_manager.cc:1905] T 67d1e4bb1c134fea8befdd288a62a6f4 P 874ee8b794ef4086a9601b6a2df8dee6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:17.302469 7403 raft_consensus.cc:2229] T 67d1e4bb1c134fea8befdd288a62a6f4 P 10148bd29f0445239c5b4e9793900f5f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:05:17.302558 7403 raft_consensus.cc:2258] T 67d1e4bb1c134fea8befdd288a62a6f4 P 10148bd29f0445239c5b4e9793900f5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:17.302743 7402 ts_tablet_manager.cc:1918] T 67d1e4bb1c134fea8befdd288a62a6f4 P 874ee8b794ef4086a9601b6a2df8dee6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 22:05:17.302832 7402 log.cc:1196] T 67d1e4bb1c134fea8befdd288a62a6f4 P 874ee8b794ef4086a9601b6a2df8dee6: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-0/wal/wals/67d1e4bb1c134fea8befdd288a62a6f4
I20260925 22:05:17.303066 7402 ts_tablet_manager.cc:1939] T 67d1e4bb1c134fea8befdd288a62a6f4 P 874ee8b794ef4086a9601b6a2df8dee6: Deleting consensus metadata
I20260925 22:05:17.303452 6838 catalog_manager.cc:4841] TS 874ee8b794ef4086a9601b6a2df8dee6 (127.5.0.65:36769): tablet 67d1e4bb1c134fea8befdd288a62a6f4 (table test-table [id=f9b04aa572ef40148138f5a1ee61e5e7]) successfully deleted
I20260925 22:05:17.303835 7403 ts_tablet_manager.cc:1905] T 67d1e4bb1c134fea8befdd288a62a6f4 P 10148bd29f0445239c5b4e9793900f5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:17.305054 7403 ts_tablet_manager.cc:1918] T 67d1e4bb1c134fea8befdd288a62a6f4 P 10148bd29f0445239c5b4e9793900f5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 22:05:17.305141 7403 log.cc:1196] T 67d1e4bb1c134fea8befdd288a62a6f4 P 10148bd29f0445239c5b4e9793900f5f: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-1/wal/wals/67d1e4bb1c134fea8befdd288a62a6f4
I20260925 22:05:17.305371 7403 ts_tablet_manager.cc:1939] T 67d1e4bb1c134fea8befdd288a62a6f4 P 10148bd29f0445239c5b4e9793900f5f: Deleting consensus metadata
I20260925 22:05:17.305761 6838 catalog_manager.cc:4841] TS 10148bd29f0445239c5b4e9793900f5f (127.5.0.66:33273): tablet 67d1e4bb1c134fea8befdd288a62a6f4 (table test-table [id=f9b04aa572ef40148138f5a1ee61e5e7]) successfully deleted
W20260925 22:05:17.306901 7338 tablet.cc:2369] T 27ecb5c0c3ba4af1b9493c20bd246145 P 38876e68cdfa46158375464729ee36f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:05:17.401971 7364 raft_consensus.cc:482] T 27ecb5c0c3ba4af1b9493c20bd246145 P 38876e68cdfa46158375464729ee36f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:05:17.402107 7364 raft_consensus.cc:504] T 27ecb5c0c3ba4af1b9493c20bd246145 P 38876e68cdfa46158375464729ee36f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:17.402480 7364 leader_election.cc:288] T 27ecb5c0c3ba4af1b9493c20bd246145 P 38876e68cdfa46158375464729ee36f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 874ee8b794ef4086a9601b6a2df8dee6 (127.5.0.65:36769), 10148bd29f0445239c5b4e9793900f5f (127.5.0.66:33273)
I20260925 22:05:17.405712 7357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:17.402762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:54679 (local address 127.5.0.65:36769)
0925 22:05:17.402812 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:17.402814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:17.402833 (+ 19us) server_negotiation.cc:407] Connection header received
0925 22:05:17.402939 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:17.402940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:17.402951 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:17.402999 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:17.403222 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.403983 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.404984 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.405351 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.405436 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:17.405449 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:17.405605 (+ 156us) server_negotiation.cc:299] Negotiation successful
0925 22:05:17.405642 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 22:05:17.405730 7230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:17.402693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:36769 (local address 127.5.0.67:54679)
0925 22:05:17.402785 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:17.402798 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:17.402848 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:17.403046 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:17.403046 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:17.403048 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:17.403142 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:17.403144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.404097 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.404098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:17.404871 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:17.404873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.405434 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.405435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:17.405525 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:17.405529 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:05:17.405598 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 22:05:17.405641 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71,"mutex_wait_us":1}
I20260925 22:05:17.406003 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27ecb5c0c3ba4af1b9493c20bd246145" candidate_uuid: "38876e68cdfa46158375464729ee36f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "874ee8b794ef4086a9601b6a2df8dee6" is_pre_election: true
I20260925 22:05:17.406134 7011 raft_consensus.cc:2454] T 27ecb5c0c3ba4af1b9493c20bd246145 P 874ee8b794ef4086a9601b6a2df8dee6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38876e68cdfa46158375464729ee36f4 in term 0.
I20260925 22:05:17.406309 7222 leader_election.cc:302] T 27ecb5c0c3ba4af1b9493c20bd246145 P 38876e68cdfa46158375464729ee36f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38876e68cdfa46158375464729ee36f4, 874ee8b794ef4086a9601b6a2df8dee6; no voters:
I20260925 22:05:17.406422 7359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:17.402932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:49665 (local address 127.5.0.66:33273)
0925 22:05:17.402969 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:17.402971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:17.402978 (+ 7us) server_negotiation.cc:407] Connection header received
0925 22:05:17.403034 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:17.403035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:17.403044 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:17.403122 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:17.403542 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.404106 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.405472 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.406038 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.406081 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:17.406090 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:17.406312 (+ 222us) server_negotiation.cc:299] Negotiation successful
0925 22:05:17.406360 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:05:17.406425 7408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:17.402796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:33273 (local address 127.5.0.67:49665)
0925 22:05:17.402910 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:17.402919 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:17.402958 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:17.403134 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:17.403135 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:17.403136 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:17.403457 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:17.403459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.404195 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.404197 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:17.405365 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:17.405368 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.406147 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.406148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:17.406220 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:17.406224 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:05:17.406297 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 22:05:17.406335 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":65,"threads_started":1}
I20260925 22:05:17.406458 7364 raft_consensus.cc:2781] T 27ecb5c0c3ba4af1b9493c20bd246145 P 38876e68cdfa46158375464729ee36f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:05:17.406512 7364 raft_consensus.cc:482] T 27ecb5c0c3ba4af1b9493c20bd246145 P 38876e68cdfa46158375464729ee36f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:05:17.406543 7364 raft_consensus.cc:3037] T 27ecb5c0c3ba4af1b9493c20bd246145 P 38876e68cdfa46158375464729ee36f4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:17.407191 7364 raft_consensus.cc:504] T 27ecb5c0c3ba4af1b9493c20bd246145 P 38876e68cdfa46158375464729ee36f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:17.407272 7364 leader_election.cc:288] T 27ecb5c0c3ba4af1b9493c20bd246145 P 38876e68cdfa46158375464729ee36f4 [CANDIDATE]: Term 1 election: Requested vote from peers 874ee8b794ef4086a9601b6a2df8dee6 (127.5.0.65:36769), 10148bd29f0445239c5b4e9793900f5f (127.5.0.66:33273)
I20260925 22:05:17.407593 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27ecb5c0c3ba4af1b9493c20bd246145" candidate_uuid: "38876e68cdfa46158375464729ee36f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "874ee8b794ef4086a9601b6a2df8dee6"
I20260925 22:05:17.407624 7150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27ecb5c0c3ba4af1b9493c20bd246145" candidate_uuid: "38876e68cdfa46158375464729ee36f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10148bd29f0445239c5b4e9793900f5f" is_pre_election: true
I20260925 22:05:17.407688 7011 raft_consensus.cc:3037] T 27ecb5c0c3ba4af1b9493c20bd246145 P 874ee8b794ef4086a9601b6a2df8dee6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:17.407709 7150 raft_consensus.cc:2454] T 27ecb5c0c3ba4af1b9493c20bd246145 P 10148bd29f0445239c5b4e9793900f5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38876e68cdfa46158375464729ee36f4 in term 0.
I20260925 22:05:17.407811 7151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27ecb5c0c3ba4af1b9493c20bd246145" candidate_uuid: "38876e68cdfa46158375464729ee36f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10148bd29f0445239c5b4e9793900f5f"
I20260925 22:05:17.407863 7151 raft_consensus.cc:3037] T 27ecb5c0c3ba4af1b9493c20bd246145 P 10148bd29f0445239c5b4e9793900f5f [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:17.408456 7151 raft_consensus.cc:2454] T 27ecb5c0c3ba4af1b9493c20bd246145 P 10148bd29f0445239c5b4e9793900f5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38876e68cdfa46158375464729ee36f4 in term 1.
I20260925 22:05:17.408490 7011 raft_consensus.cc:2454] T 27ecb5c0c3ba4af1b9493c20bd246145 P 874ee8b794ef4086a9601b6a2df8dee6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38876e68cdfa46158375464729ee36f4 in term 1.
I20260925 22:05:17.408658 7222 leader_election.cc:302] T 27ecb5c0c3ba4af1b9493c20bd246145 P 38876e68cdfa46158375464729ee36f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38876e68cdfa46158375464729ee36f4, 874ee8b794ef4086a9601b6a2df8dee6; no voters:
I20260925 22:05:17.408790 7364 raft_consensus.cc:2781] T 27ecb5c0c3ba4af1b9493c20bd246145 P 38876e68cdfa46158375464729ee36f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:17.408962 7364 raft_consensus.cc:686] T 27ecb5c0c3ba4af1b9493c20bd246145 P 38876e68cdfa46158375464729ee36f4 [term 1 LEADER]: Becoming Leader. State: Replica: 38876e68cdfa46158375464729ee36f4, State: Running, Role: LEADER
I20260925 22:05:17.409034 7364 consensus_queue.cc:237] T 27ecb5c0c3ba4af1b9493c20bd246145 P 38876e68cdfa46158375464729ee36f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:17.409758 6853 catalog_manager.cc:5495] T 27ecb5c0c3ba4af1b9493c20bd246145 P 38876e68cdfa46158375464729ee36f4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 38876e68cdfa46158375464729ee36f4 (127.5.0.67). New cstate: current_term: 1 leader_uuid: "38876e68cdfa46158375464729ee36f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } health_report { overall_health: HEALTHY } } }
I20260925 22:05:17.445902 7356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:17.443141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37290 (local address 127.5.0.67:37045)
0925 22:05:17.443188 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:17.443190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:17.443228 (+ 38us) server_negotiation.cc:407] Connection header received
0925 22:05:17.443314 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:17.443315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:17.443338 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:17.443420 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:05:17.443516 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.444289 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.445047 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.445329 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.445407 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:17.445531 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:05:17.445680 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:05:17.445766 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:17.445788 (+ 22us) server_negotiation.cc:299] Negotiation successful
0925 22:05:17.445819 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 22:05:17.451138 7357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:17.448687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:50997 (local address 127.5.0.65:36769)
0925 22:05:17.448731 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:17.448732 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:17.448743 (+ 11us) server_negotiation.cc:407] Connection header received
0925 22:05:17.448817 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:17.448818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:17.448826 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:17.448874 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:05:17.449011 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.449446 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.450506 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.450675 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.450728 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:17.450920 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:05:17.450944 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:05:17.450978 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:17.451062 (+ 84us) server_negotiation.cc:299] Negotiation successful
0925 22:05:17.451080 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 22:05:17.451177 7343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:17.448626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:36769 (local address 127.5.0.67:50997)
0925 22:05:17.448707 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:17.448718 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:17.448759 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:17.448890 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:17.448891 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:17.448892 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 22:05:17.448948 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:17.448950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.449531 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.449532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:17.450426 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:17.450429 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.450749 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.450750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:17.450818 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:17.450832 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 22:05:17.451005 (+ 173us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 22:05:17.451007 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 22:05:17.451061 (+ 54us) client_negotiation.cc:240] Negotiation successful
0925 22:05:17.451102 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260925 22:05:17.453214 7151 raft_consensus.cc:1261] T 27ecb5c0c3ba4af1b9493c20bd246145 P 10148bd29f0445239c5b4e9793900f5f [term 1 FOLLOWER]: Refusing update from remote peer 38876e68cdfa46158375464729ee36f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:05:17.453378 7011 raft_consensus.cc:1261] T 27ecb5c0c3ba4af1b9493c20bd246145 P 874ee8b794ef4086a9601b6a2df8dee6 [term 1 FOLLOWER]: Refusing update from remote peer 38876e68cdfa46158375464729ee36f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:05:17.453565 7409 consensus_queue.cc:1035] T 27ecb5c0c3ba4af1b9493c20bd246145 P 38876e68cdfa46158375464729ee36f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:05:17.453717 7364 consensus_queue.cc:1035] T 27ecb5c0c3ba4af1b9493c20bd246145 P 38876e68cdfa46158375464729ee36f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:05:17.461205 7396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:17.458514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:37045 (local address 127.5.0.65:58541)
0925 22:05:17.458643 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:17.458655 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:17.458700 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:17.458886 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:17.458887 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:17.458888 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 22:05:17.458924 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:17.458925 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.459729 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.459730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:17.460356 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:17.460358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.460800 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.460801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:17.460865 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:17.460872 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 22:05:17.461037 (+ 165us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 22:05:17.461038 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 22:05:17.461088 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 22:05:17.461133 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109}
I20260925 22:05:17.461197 7356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:17.458562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:58541 (local address 127.5.0.67:37045)
0925 22:05:17.458788 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:17.458790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:17.458798 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:05:17.458817 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:17.458818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:17.458829 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:17.458881 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:05:17.459006 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.459661 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.460451 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:17.460718 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:17.460802 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:17.460949 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:05:17.460975 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:05:17.461036 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:17.461089 (+ 53us) server_negotiation.cc:299] Negotiation successful
0925 22:05:17.461109 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208}
W20260925 22:05:17.461711 7387 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260925 22:05:18.981297 6853 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:05:18.981395 6853 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:05:18.982160 6853 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 196ba701dd2e4bb9a12bc02d704b73ce: Sending DeleteTablet for 3 replicas of tablet 27ecb5c0c3ba4af1b9493c20bd246145
I20260925 22:05:18.982414 7271 tablet_service.cc:1514] Processing DeleteTablet for tablet 27ecb5c0c3ba4af1b9493c20bd246145 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:18 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:37280
I20260925 22:05:18.982465 6991 tablet_service.cc:1514] Processing DeleteTablet for tablet 27ecb5c0c3ba4af1b9493c20bd246145 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:18 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:44022
I20260925 22:05:18.982475 7131 tablet_service.cc:1514] Processing DeleteTablet for tablet 27ecb5c0c3ba4af1b9493c20bd246145 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:18 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:42444
I20260925 22:05:18.982602 7439 tablet_replica.cc:330] T 27ecb5c0c3ba4af1b9493c20bd246145 P 38876e68cdfa46158375464729ee36f4: stopping tablet replica
I20260925 22:05:18.982672 7441 tablet_replica.cc:330] T 27ecb5c0c3ba4af1b9493c20bd246145 P 10148bd29f0445239c5b4e9793900f5f: stopping tablet replica
I20260925 22:05:18.982720 7441 raft_consensus.cc:2229] T 27ecb5c0c3ba4af1b9493c20bd246145 P 10148bd29f0445239c5b4e9793900f5f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:05:18.982721 7439 raft_consensus.cc:2229] T 27ecb5c0c3ba4af1b9493c20bd246145 P 38876e68cdfa46158375464729ee36f4 [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:05:18.982755 7441 raft_consensus.cc:2258] T 27ecb5c0c3ba4af1b9493c20bd246145 P 10148bd29f0445239c5b4e9793900f5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:18.982810 7439 raft_consensus.cc:2258] T 27ecb5c0c3ba4af1b9493c20bd246145 P 38876e68cdfa46158375464729ee36f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:18.983029 7439 ts_tablet_manager.cc:1905] T 27ecb5c0c3ba4af1b9493c20bd246145 P 38876e68cdfa46158375464729ee36f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:18.983698 7441 ts_tablet_manager.cc:1905] T 27ecb5c0c3ba4af1b9493c20bd246145 P 10148bd29f0445239c5b4e9793900f5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:18.984967 7439 ts_tablet_manager.cc:1918] T 27ecb5c0c3ba4af1b9493c20bd246145 P 38876e68cdfa46158375464729ee36f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 22:05:18.985018 7439 log.cc:1196] T 27ecb5c0c3ba4af1b9493c20bd246145 P 38876e68cdfa46158375464729ee36f4: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-2/wal/wals/27ecb5c0c3ba4af1b9493c20bd246145
I20260925 22:05:18.985256 7439 ts_tablet_manager.cc:1939] T 27ecb5c0c3ba4af1b9493c20bd246145 P 38876e68cdfa46158375464729ee36f4: Deleting consensus metadata
I20260925 22:05:18.985447 7441 ts_tablet_manager.cc:1918] T 27ecb5c0c3ba4af1b9493c20bd246145 P 10148bd29f0445239c5b4e9793900f5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 22:05:18.985507 7441 log.cc:1196] T 27ecb5c0c3ba4af1b9493c20bd246145 P 10148bd29f0445239c5b4e9793900f5f: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-1/wal/wals/27ecb5c0c3ba4af1b9493c20bd246145
I20260925 22:05:18.982905 7440 tablet_replica.cc:330] T 27ecb5c0c3ba4af1b9493c20bd246145 P 874ee8b794ef4086a9601b6a2df8dee6: stopping tablet replica
I20260925 22:05:18.985596 6839 catalog_manager.cc:4841] TS 38876e68cdfa46158375464729ee36f4 (127.5.0.67:37045): tablet 27ecb5c0c3ba4af1b9493c20bd246145 (table test-table [id=7ca49adde8e84c03bb9d1e195c6028bb]) successfully deleted
I20260925 22:05:18.985625 7440 raft_consensus.cc:2229] T 27ecb5c0c3ba4af1b9493c20bd246145 P 874ee8b794ef4086a9601b6a2df8dee6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:05:18.985667 7440 raft_consensus.cc:2258] T 27ecb5c0c3ba4af1b9493c20bd246145 P 874ee8b794ef4086a9601b6a2df8dee6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:18.985765 7441 ts_tablet_manager.cc:1939] T 27ecb5c0c3ba4af1b9493c20bd246145 P 10148bd29f0445239c5b4e9793900f5f: Deleting consensus metadata
I20260925 22:05:18.985921 7440 ts_tablet_manager.cc:1905] T 27ecb5c0c3ba4af1b9493c20bd246145 P 874ee8b794ef4086a9601b6a2df8dee6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:18.986073 6838 catalog_manager.cc:4841] TS 10148bd29f0445239c5b4e9793900f5f (127.5.0.66:33273): tablet 27ecb5c0c3ba4af1b9493c20bd246145 (table test-table [id=7ca49adde8e84c03bb9d1e195c6028bb]) successfully deleted
I20260925 22:05:18.987300 7440 ts_tablet_manager.cc:1918] T 27ecb5c0c3ba4af1b9493c20bd246145 P 874ee8b794ef4086a9601b6a2df8dee6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 22:05:18.987383 7440 log.cc:1196] T 27ecb5c0c3ba4af1b9493c20bd246145 P 874ee8b794ef4086a9601b6a2df8dee6: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-0/wal/wals/27ecb5c0c3ba4af1b9493c20bd246145
I20260925 22:05:18.987594 7440 ts_tablet_manager.cc:1939] T 27ecb5c0c3ba4af1b9493c20bd246145 P 874ee8b794ef4086a9601b6a2df8dee6: Deleting consensus metadata
I20260925 22:05:18.987969 6838 catalog_manager.cc:4841] TS 874ee8b794ef4086a9601b6a2df8dee6 (127.5.0.65:36769): tablet 27ecb5c0c3ba4af1b9493c20bd246145 (table test-table [id=7ca49adde8e84c03bb9d1e195c6028bb]) successfully deleted
Sep 25 22:05:18 dist-test-slave-7mqr krb5kdc[6809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373918, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 22:05:18 dist-test-slave-7mqr krb5kdc[6809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373918, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:19.006183 7449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:18.999116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50446 (local address 127.5.0.126:34405)
0925 22:05:18.999285 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:18.999287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:18.999313 (+ 26us) server_negotiation.cc:407] Connection header received
0925 22:05:18.999335 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:18.999336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:18.999346 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:18.999397 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:18.999994 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:19.000495 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:19.001215 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:19.001401 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:19.001452 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:19.003844 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:19.003845 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:19.003845 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:19.003852 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:19.005251 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:19.005553 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:19.005554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:19.005555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:19.005604 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:19.005764 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:19.005764 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:19.005765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:19.005865 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:19.005939 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:19.006022 (+ 83us) server_negotiation.cc:299] Negotiation successful
0925 22:05:19.006082 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":105,"threads_started":1}
I20260925 22:05:19.007599 6853 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50446:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:05:19.007752 6853 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:05:19.009178 6991 tablet_service.cc:1467] Processing CreateTablet for tablet cf0af9e189bf49619127c95b0cdea0e3 (DEFAULT_TABLE table=test-table [id=aa182b6911384a2daed51c2d7a16a4ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:19.009189 7271 tablet_service.cc:1467] Processing CreateTablet for tablet cf0af9e189bf49619127c95b0cdea0e3 (DEFAULT_TABLE table=test-table [id=aa182b6911384a2daed51c2d7a16a4ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:19.009227 7131 tablet_service.cc:1467] Processing CreateTablet for tablet cf0af9e189bf49619127c95b0cdea0e3 (DEFAULT_TABLE table=test-table [id=aa182b6911384a2daed51c2d7a16a4ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:19.009337 7131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf0af9e189bf49619127c95b0cdea0e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:19.009337 7271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf0af9e189bf49619127c95b0cdea0e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:19.009338 6991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf0af9e189bf49619127c95b0cdea0e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:19.010804 7452 tablet_bootstrap.cc:492] T cf0af9e189bf49619127c95b0cdea0e3 P 874ee8b794ef4086a9601b6a2df8dee6: Bootstrap starting.
I20260925 22:05:19.010900 7451 tablet_bootstrap.cc:492] T cf0af9e189bf49619127c95b0cdea0e3 P 10148bd29f0445239c5b4e9793900f5f: Bootstrap starting.
I20260925 22:05:19.011184 7452 tablet_bootstrap.cc:654] T cf0af9e189bf49619127c95b0cdea0e3 P 874ee8b794ef4086a9601b6a2df8dee6: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:19.011281 7451 tablet_bootstrap.cc:654] T cf0af9e189bf49619127c95b0cdea0e3 P 10148bd29f0445239c5b4e9793900f5f: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:19.011618 7450 tablet_bootstrap.cc:492] T cf0af9e189bf49619127c95b0cdea0e3 P 38876e68cdfa46158375464729ee36f4: Bootstrap starting.
I20260925 22:05:19.011842 7452 tablet_bootstrap.cc:492] T cf0af9e189bf49619127c95b0cdea0e3 P 874ee8b794ef4086a9601b6a2df8dee6: No bootstrap required, opened a new log
I20260925 22:05:19.011924 7452 ts_tablet_manager.cc:1397] T cf0af9e189bf49619127c95b0cdea0e3 P 874ee8b794ef4086a9601b6a2df8dee6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 22:05:19.011984 7450 tablet_bootstrap.cc:654] T cf0af9e189bf49619127c95b0cdea0e3 P 38876e68cdfa46158375464729ee36f4: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:19.012048 7452 raft_consensus.cc:348] T cf0af9e189bf49619127c95b0cdea0e3 P 874ee8b794ef4086a9601b6a2df8dee6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:19.012122 7452 raft_consensus.cc:374] T cf0af9e189bf49619127c95b0cdea0e3 P 874ee8b794ef4086a9601b6a2df8dee6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:19.012148 7452 raft_consensus.cc:729] T cf0af9e189bf49619127c95b0cdea0e3 P 874ee8b794ef4086a9601b6a2df8dee6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 874ee8b794ef4086a9601b6a2df8dee6, State: Initialized, Role: FOLLOWER
I20260925 22:05:19.012198 7452 consensus_queue.cc:260] T cf0af9e189bf49619127c95b0cdea0e3 P 874ee8b794ef4086a9601b6a2df8dee6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:19.012292 7452 ts_tablet_manager.cc:1428] T cf0af9e189bf49619127c95b0cdea0e3 P 874ee8b794ef4086a9601b6a2df8dee6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:19.012632 7451 tablet_bootstrap.cc:492] T cf0af9e189bf49619127c95b0cdea0e3 P 10148bd29f0445239c5b4e9793900f5f: No bootstrap required, opened a new log
I20260925 22:05:19.012687 7451 ts_tablet_manager.cc:1397] T cf0af9e189bf49619127c95b0cdea0e3 P 10148bd29f0445239c5b4e9793900f5f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:05:19.012971 7450 tablet_bootstrap.cc:492] T cf0af9e189bf49619127c95b0cdea0e3 P 38876e68cdfa46158375464729ee36f4: No bootstrap required, opened a new log
I20260925 22:05:19.013039 7450 ts_tablet_manager.cc:1397] T cf0af9e189bf49619127c95b0cdea0e3 P 38876e68cdfa46158375464729ee36f4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 22:05:19.013159 7450 raft_consensus.cc:348] T cf0af9e189bf49619127c95b0cdea0e3 P 38876e68cdfa46158375464729ee36f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:19.013223 7450 raft_consensus.cc:374] T cf0af9e189bf49619127c95b0cdea0e3 P 38876e68cdfa46158375464729ee36f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:19.013237 7450 raft_consensus.cc:729] T cf0af9e189bf49619127c95b0cdea0e3 P 38876e68cdfa46158375464729ee36f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38876e68cdfa46158375464729ee36f4, State: Initialized, Role: FOLLOWER
I20260925 22:05:19.013290 7450 consensus_queue.cc:260] T cf0af9e189bf49619127c95b0cdea0e3 P 38876e68cdfa46158375464729ee36f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:19.013576 7451 raft_consensus.cc:348] T cf0af9e189bf49619127c95b0cdea0e3 P 10148bd29f0445239c5b4e9793900f5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:19.013660 7451 raft_consensus.cc:374] T cf0af9e189bf49619127c95b0cdea0e3 P 10148bd29f0445239c5b4e9793900f5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:19.013685 7451 raft_consensus.cc:729] T cf0af9e189bf49619127c95b0cdea0e3 P 10148bd29f0445239c5b4e9793900f5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10148bd29f0445239c5b4e9793900f5f, State: Initialized, Role: FOLLOWER
I20260925 22:05:19.013769 7451 consensus_queue.cc:260] T cf0af9e189bf49619127c95b0cdea0e3 P 10148bd29f0445239c5b4e9793900f5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:19.013954 7451 ts_tablet_manager.cc:1428] T cf0af9e189bf49619127c95b0cdea0e3 P 10148bd29f0445239c5b4e9793900f5f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260925 22:05:19.014039 7450 ts_tablet_manager.cc:1428] T cf0af9e189bf49619127c95b0cdea0e3 P 38876e68cdfa46158375464729ee36f4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260925 22:05:19.088485 7370 raft_consensus.cc:482] T cf0af9e189bf49619127c95b0cdea0e3 P 874ee8b794ef4086a9601b6a2df8dee6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:05:19.088587 7370 raft_consensus.cc:504] T cf0af9e189bf49619127c95b0cdea0e3 P 874ee8b794ef4086a9601b6a2df8dee6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:19.088780 7370 leader_election.cc:288] T cf0af9e189bf49619127c95b0cdea0e3 P 874ee8b794ef4086a9601b6a2df8dee6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10148bd29f0445239c5b4e9793900f5f (127.5.0.66:33273), 38876e68cdfa46158375464729ee36f4 (127.5.0.67:37045)
I20260925 22:05:19.089015 7151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf0af9e189bf49619127c95b0cdea0e3" candidate_uuid: "874ee8b794ef4086a9601b6a2df8dee6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10148bd29f0445239c5b4e9793900f5f" is_pre_election: true
I20260925 22:05:19.089015 7291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf0af9e189bf49619127c95b0cdea0e3" candidate_uuid: "874ee8b794ef4086a9601b6a2df8dee6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38876e68cdfa46158375464729ee36f4" is_pre_election: true
I20260925 22:05:19.089126 7151 raft_consensus.cc:2454] T cf0af9e189bf49619127c95b0cdea0e3 P 10148bd29f0445239c5b4e9793900f5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 874ee8b794ef4086a9601b6a2df8dee6 in term 0.
I20260925 22:05:19.089126 7291 raft_consensus.cc:2454] T cf0af9e189bf49619127c95b0cdea0e3 P 38876e68cdfa46158375464729ee36f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 874ee8b794ef4086a9601b6a2df8dee6 in term 0.
I20260925 22:05:19.089313 6943 leader_election.cc:302] T cf0af9e189bf49619127c95b0cdea0e3 P 874ee8b794ef4086a9601b6a2df8dee6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38876e68cdfa46158375464729ee36f4, 874ee8b794ef4086a9601b6a2df8dee6; no voters:
I20260925 22:05:19.089454 7370 raft_consensus.cc:2781] T cf0af9e189bf49619127c95b0cdea0e3 P 874ee8b794ef4086a9601b6a2df8dee6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:05:19.089509 7370 raft_consensus.cc:482] T cf0af9e189bf49619127c95b0cdea0e3 P 874ee8b794ef4086a9601b6a2df8dee6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:05:19.089530 7370 raft_consensus.cc:3037] T cf0af9e189bf49619127c95b0cdea0e3 P 874ee8b794ef4086a9601b6a2df8dee6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:19.090248 7370 raft_consensus.cc:504] T cf0af9e189bf49619127c95b0cdea0e3 P 874ee8b794ef4086a9601b6a2df8dee6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:19.090378 7370 leader_election.cc:288] T cf0af9e189bf49619127c95b0cdea0e3 P 874ee8b794ef4086a9601b6a2df8dee6 [CANDIDATE]: Term 1 election: Requested vote from peers 10148bd29f0445239c5b4e9793900f5f (127.5.0.66:33273), 38876e68cdfa46158375464729ee36f4 (127.5.0.67:37045)
I20260925 22:05:19.090508 7291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf0af9e189bf49619127c95b0cdea0e3" candidate_uuid: "874ee8b794ef4086a9601b6a2df8dee6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38876e68cdfa46158375464729ee36f4"
I20260925 22:05:19.090595 7291 raft_consensus.cc:3037] T cf0af9e189bf49619127c95b0cdea0e3 P 38876e68cdfa46158375464729ee36f4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:19.090590 7151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf0af9e189bf49619127c95b0cdea0e3" candidate_uuid: "874ee8b794ef4086a9601b6a2df8dee6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10148bd29f0445239c5b4e9793900f5f"
I20260925 22:05:19.090655 7151 raft_consensus.cc:3037] T cf0af9e189bf49619127c95b0cdea0e3 P 10148bd29f0445239c5b4e9793900f5f [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:19.091357 7151 raft_consensus.cc:2454] T cf0af9e189bf49619127c95b0cdea0e3 P 10148bd29f0445239c5b4e9793900f5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 874ee8b794ef4086a9601b6a2df8dee6 in term 1.
I20260925 22:05:19.091362 7291 raft_consensus.cc:2454] T cf0af9e189bf49619127c95b0cdea0e3 P 38876e68cdfa46158375464729ee36f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 874ee8b794ef4086a9601b6a2df8dee6 in term 1.
I20260925 22:05:19.091517 6943 leader_election.cc:302] T cf0af9e189bf49619127c95b0cdea0e3 P 874ee8b794ef4086a9601b6a2df8dee6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38876e68cdfa46158375464729ee36f4, 874ee8b794ef4086a9601b6a2df8dee6; no voters:
I20260925 22:05:19.091655 7370 raft_consensus.cc:2781] T cf0af9e189bf49619127c95b0cdea0e3 P 874ee8b794ef4086a9601b6a2df8dee6 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:19.091717 7370 raft_consensus.cc:686] T cf0af9e189bf49619127c95b0cdea0e3 P 874ee8b794ef4086a9601b6a2df8dee6 [term 1 LEADER]: Becoming Leader. State: Replica: 874ee8b794ef4086a9601b6a2df8dee6, State: Running, Role: LEADER
I20260925 22:05:19.091789 7370 consensus_queue.cc:237] T cf0af9e189bf49619127c95b0cdea0e3 P 874ee8b794ef4086a9601b6a2df8dee6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:19.092444 6854 catalog_manager.cc:5495] T cf0af9e189bf49619127c95b0cdea0e3 P 874ee8b794ef4086a9601b6a2df8dee6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 874ee8b794ef4086a9601b6a2df8dee6 (127.5.0.65). New cstate: current_term: 1 leader_uuid: "874ee8b794ef4086a9601b6a2df8dee6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } health_report { overall_health: UNKNOWN } } }
I20260925 22:05:19.106276 7459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:19.104190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44036 (local address 127.5.0.65:36769)
0925 22:05:19.104320 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:19.104322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:19.104329 (+ 7us) server_negotiation.cc:407] Connection header received
0925 22:05:19.104351 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:19.104352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:19.104368 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:19.104436 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:05:19.104540 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:19.105004 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:19.105724 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:19.105911 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:19.105965 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:19.106081 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:05:19.106108 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:05:19.106157 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:19.106202 (+ 45us) server_negotiation.cc:299] Negotiation successful
0925 22:05:19.106217 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":89,"threads_started":1}
I20260925 22:05:19.108578 7151 raft_consensus.cc:1261] T cf0af9e189bf49619127c95b0cdea0e3 P 10148bd29f0445239c5b4e9793900f5f [term 1 FOLLOWER]: Refusing update from remote peer 874ee8b794ef4086a9601b6a2df8dee6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:05:19.108592 7291 raft_consensus.cc:1261] T cf0af9e189bf49619127c95b0cdea0e3 P 38876e68cdfa46158375464729ee36f4 [term 1 FOLLOWER]: Refusing update from remote peer 874ee8b794ef4086a9601b6a2df8dee6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:05:19.108831 7370 consensus_queue.cc:1035] T cf0af9e189bf49619127c95b0cdea0e3 P 874ee8b794ef4086a9601b6a2df8dee6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:05:19.108954 7425 consensus_queue.cc:1035] T cf0af9e189bf49619127c95b0cdea0e3 P 874ee8b794ef4086a9601b6a2df8dee6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:05:19.119472 6853 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:05:19.119561 6853 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:05:19.120174 6853 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 196ba701dd2e4bb9a12bc02d704b73ce: Sending DeleteTablet for 3 replicas of tablet cf0af9e189bf49619127c95b0cdea0e3
I20260925 22:05:19.120448 7271 tablet_service.cc:1514] Processing DeleteTablet for tablet cf0af9e189bf49619127c95b0cdea0e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:19 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:37280
I20260925 22:05:19.120463 7131 tablet_service.cc:1514] Processing DeleteTablet for tablet cf0af9e189bf49619127c95b0cdea0e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:19 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:42444
I20260925 22:05:19.120558 7441 tablet_replica.cc:330] T cf0af9e189bf49619127c95b0cdea0e3 P 10148bd29f0445239c5b4e9793900f5f: stopping tablet replica
I20260925 22:05:19.120558 7439 tablet_replica.cc:330] T cf0af9e189bf49619127c95b0cdea0e3 P 38876e68cdfa46158375464729ee36f4: stopping tablet replica
I20260925 22:05:19.120556 6991 tablet_service.cc:1514] Processing DeleteTablet for tablet cf0af9e189bf49619127c95b0cdea0e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:19 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:44022
I20260925 22:05:19.120611 7439 raft_consensus.cc:2229] T cf0af9e189bf49619127c95b0cdea0e3 P 38876e68cdfa46158375464729ee36f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:05:19.120604 7441 raft_consensus.cc:2229] T cf0af9e189bf49619127c95b0cdea0e3 P 10148bd29f0445239c5b4e9793900f5f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:05:19.120646 7439 raft_consensus.cc:2258] T cf0af9e189bf49619127c95b0cdea0e3 P 38876e68cdfa46158375464729ee36f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:19.120644 7440 tablet_replica.cc:330] T cf0af9e189bf49619127c95b0cdea0e3 P 874ee8b794ef4086a9601b6a2df8dee6: stopping tablet replica
I20260925 22:05:19.120648 7441 raft_consensus.cc:2258] T cf0af9e189bf49619127c95b0cdea0e3 P 10148bd29f0445239c5b4e9793900f5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:19.120694 7440 raft_consensus.cc:2229] T cf0af9e189bf49619127c95b0cdea0e3 P 874ee8b794ef4086a9601b6a2df8dee6 [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:05:19.120671 6853 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50446:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 22:05:19.120759 7440 raft_consensus.cc:2258] T cf0af9e189bf49619127c95b0cdea0e3 P 874ee8b794ef4086a9601b6a2df8dee6 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 22:05:19.120802 6853 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:05:19.120990 7439 ts_tablet_manager.cc:1905] T cf0af9e189bf49619127c95b0cdea0e3 P 38876e68cdfa46158375464729ee36f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:19.121114 7440 ts_tablet_manager.cc:1905] T cf0af9e189bf49619127c95b0cdea0e3 P 874ee8b794ef4086a9601b6a2df8dee6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:19.121961 7271 tablet_service.cc:1467] Processing CreateTablet for tablet 9ec890372ccf4bfa975037a7f6e18399 (DEFAULT_TABLE table=test-table [id=a2fbd548bdbb43538197c33380c0b968]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:19.122074 7271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ec890372ccf4bfa975037a7f6e18399. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:19.122611 6991 tablet_service.cc:1467] Processing CreateTablet for tablet 9ec890372ccf4bfa975037a7f6e18399 (DEFAULT_TABLE table=test-table [id=a2fbd548bdbb43538197c33380c0b968]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:19.122658 7131 tablet_service.cc:1467] Processing CreateTablet for tablet 9ec890372ccf4bfa975037a7f6e18399 (DEFAULT_TABLE table=test-table [id=a2fbd548bdbb43538197c33380c0b968]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:19.122816 7131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ec890372ccf4bfa975037a7f6e18399. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:19.122843 7441 ts_tablet_manager.cc:1905] T cf0af9e189bf49619127c95b0cdea0e3 P 10148bd29f0445239c5b4e9793900f5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:19.124078 7451 tablet_bootstrap.cc:492] T 9ec890372ccf4bfa975037a7f6e18399 P 10148bd29f0445239c5b4e9793900f5f: Bootstrap starting.
I20260925 22:05:19.124126 7439 ts_tablet_manager.cc:1918] T cf0af9e189bf49619127c95b0cdea0e3 P 38876e68cdfa46158375464729ee36f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 22:05:19.124178 7439 log.cc:1196] T cf0af9e189bf49619127c95b0cdea0e3 P 38876e68cdfa46158375464729ee36f4: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-2/wal/wals/cf0af9e189bf49619127c95b0cdea0e3
I20260925 22:05:19.124397 7439 ts_tablet_manager.cc:1939] T cf0af9e189bf49619127c95b0cdea0e3 P 38876e68cdfa46158375464729ee36f4: Deleting consensus metadata
I20260925 22:05:19.124409 7451 tablet_bootstrap.cc:654] T 9ec890372ccf4bfa975037a7f6e18399 P 10148bd29f0445239c5b4e9793900f5f: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:19.124950 7451 tablet_bootstrap.cc:492] T 9ec890372ccf4bfa975037a7f6e18399 P 10148bd29f0445239c5b4e9793900f5f: No bootstrap required, opened a new log
I20260925 22:05:19.124981 7451 ts_tablet_manager.cc:1397] T 9ec890372ccf4bfa975037a7f6e18399 P 10148bd29f0445239c5b4e9793900f5f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 22:05:19.125048 7450 tablet_bootstrap.cc:492] T 9ec890372ccf4bfa975037a7f6e18399 P 38876e68cdfa46158375464729ee36f4: Bootstrap starting.
I20260925 22:05:19.125085 7451 raft_consensus.cc:348] T 9ec890372ccf4bfa975037a7f6e18399 P 10148bd29f0445239c5b4e9793900f5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:19.125128 7451 raft_consensus.cc:374] T 9ec890372ccf4bfa975037a7f6e18399 P 10148bd29f0445239c5b4e9793900f5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:19.125141 7451 raft_consensus.cc:729] T 9ec890372ccf4bfa975037a7f6e18399 P 10148bd29f0445239c5b4e9793900f5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10148bd29f0445239c5b4e9793900f5f, State: Initialized, Role: FOLLOWER
I20260925 22:05:19.125150 6839 catalog_manager.cc:4841] TS 38876e68cdfa46158375464729ee36f4 (127.5.0.67:37045): tablet cf0af9e189bf49619127c95b0cdea0e3 (table test-table [id=aa182b6911384a2daed51c2d7a16a4ed]) successfully deleted
I20260925 22:05:19.125204 7451 consensus_queue.cc:260] T 9ec890372ccf4bfa975037a7f6e18399 P 10148bd29f0445239c5b4e9793900f5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:19.125310 7451 ts_tablet_manager.cc:1428] T 9ec890372ccf4bfa975037a7f6e18399 P 10148bd29f0445239c5b4e9793900f5f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:19.125361 7441 ts_tablet_manager.cc:1918] T cf0af9e189bf49619127c95b0cdea0e3 P 10148bd29f0445239c5b4e9793900f5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 22:05:19.125401 7450 tablet_bootstrap.cc:654] T 9ec890372ccf4bfa975037a7f6e18399 P 38876e68cdfa46158375464729ee36f4: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:19.125411 7441 log.cc:1196] T cf0af9e189bf49619127c95b0cdea0e3 P 10148bd29f0445239c5b4e9793900f5f: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-1/wal/wals/cf0af9e189bf49619127c95b0cdea0e3
I20260925 22:05:19.125633 7441 ts_tablet_manager.cc:1939] T cf0af9e189bf49619127c95b0cdea0e3 P 10148bd29f0445239c5b4e9793900f5f: Deleting consensus metadata
I20260925 22:05:19.125934 7450 tablet_bootstrap.cc:492] T 9ec890372ccf4bfa975037a7f6e18399 P 38876e68cdfa46158375464729ee36f4: No bootstrap required, opened a new log
I20260925 22:05:19.125911 6838 catalog_manager.cc:4841] TS 10148bd29f0445239c5b4e9793900f5f (127.5.0.66:33273): tablet cf0af9e189bf49619127c95b0cdea0e3 (table test-table [id=aa182b6911384a2daed51c2d7a16a4ed]) successfully deleted
I20260925 22:05:19.125985 7450 ts_tablet_manager.cc:1397] T 9ec890372ccf4bfa975037a7f6e18399 P 38876e68cdfa46158375464729ee36f4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 22:05:19.126024 7440 ts_tablet_manager.cc:1918] T cf0af9e189bf49619127c95b0cdea0e3 P 874ee8b794ef4086a9601b6a2df8dee6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 22:05:19.126068 7440 log.cc:1196] T cf0af9e189bf49619127c95b0cdea0e3 P 874ee8b794ef4086a9601b6a2df8dee6: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-0/wal/wals/cf0af9e189bf49619127c95b0cdea0e3
I20260925 22:05:19.126086 7450 raft_consensus.cc:348] T 9ec890372ccf4bfa975037a7f6e18399 P 38876e68cdfa46158375464729ee36f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:19.126171 7450 raft_consensus.cc:374] T 9ec890372ccf4bfa975037a7f6e18399 P 38876e68cdfa46158375464729ee36f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:19.126196 7450 raft_consensus.cc:729] T 9ec890372ccf4bfa975037a7f6e18399 P 38876e68cdfa46158375464729ee36f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38876e68cdfa46158375464729ee36f4, State: Initialized, Role: FOLLOWER
I20260925 22:05:19.126245 7450 consensus_queue.cc:260] T 9ec890372ccf4bfa975037a7f6e18399 P 38876e68cdfa46158375464729ee36f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:19.126277 7440 ts_tablet_manager.cc:1939] T cf0af9e189bf49619127c95b0cdea0e3 P 874ee8b794ef4086a9601b6a2df8dee6: Deleting consensus metadata
I20260925 22:05:19.126343 7450 ts_tablet_manager.cc:1428] T 9ec890372ccf4bfa975037a7f6e18399 P 38876e68cdfa46158375464729ee36f4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:19.126410 6991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ec890372ccf4bfa975037a7f6e18399. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:19.126775 6838 catalog_manager.cc:4841] TS 874ee8b794ef4086a9601b6a2df8dee6 (127.5.0.65:36769): tablet cf0af9e189bf49619127c95b0cdea0e3 (table test-table [id=aa182b6911384a2daed51c2d7a16a4ed]) successfully deleted
I20260925 22:05:19.127431 7452 tablet_bootstrap.cc:492] T 9ec890372ccf4bfa975037a7f6e18399 P 874ee8b794ef4086a9601b6a2df8dee6: Bootstrap starting.
I20260925 22:05:19.127766 7452 tablet_bootstrap.cc:654] T 9ec890372ccf4bfa975037a7f6e18399 P 874ee8b794ef4086a9601b6a2df8dee6: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:19.128321 7452 tablet_bootstrap.cc:492] T 9ec890372ccf4bfa975037a7f6e18399 P 874ee8b794ef4086a9601b6a2df8dee6: No bootstrap required, opened a new log
I20260925 22:05:19.128360 7452 ts_tablet_manager.cc:1397] T 9ec890372ccf4bfa975037a7f6e18399 P 874ee8b794ef4086a9601b6a2df8dee6: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 22:05:19.128469 7452 raft_consensus.cc:348] T 9ec890372ccf4bfa975037a7f6e18399 P 874ee8b794ef4086a9601b6a2df8dee6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:19.128538 7452 raft_consensus.cc:374] T 9ec890372ccf4bfa975037a7f6e18399 P 874ee8b794ef4086a9601b6a2df8dee6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:19.128566 7452 raft_consensus.cc:729] T 9ec890372ccf4bfa975037a7f6e18399 P 874ee8b794ef4086a9601b6a2df8dee6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 874ee8b794ef4086a9601b6a2df8dee6, State: Initialized, Role: FOLLOWER
I20260925 22:05:19.128618 7452 consensus_queue.cc:260] T 9ec890372ccf4bfa975037a7f6e18399 P 874ee8b794ef4086a9601b6a2df8dee6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:19.128733 7452 ts_tablet_manager.cc:1428] T 9ec890372ccf4bfa975037a7f6e18399 P 874ee8b794ef4086a9601b6a2df8dee6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:19.193099 7430 raft_consensus.cc:482] T 9ec890372ccf4bfa975037a7f6e18399 P 38876e68cdfa46158375464729ee36f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:05:19.193218 7430 raft_consensus.cc:504] T 9ec890372ccf4bfa975037a7f6e18399 P 38876e68cdfa46158375464729ee36f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:19.193403 7430 leader_election.cc:288] T 9ec890372ccf4bfa975037a7f6e18399 P 38876e68cdfa46158375464729ee36f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 874ee8b794ef4086a9601b6a2df8dee6 (127.5.0.65:36769), 10148bd29f0445239c5b4e9793900f5f (127.5.0.66:33273)
I20260925 22:05:19.193651 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ec890372ccf4bfa975037a7f6e18399" candidate_uuid: "38876e68cdfa46158375464729ee36f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "874ee8b794ef4086a9601b6a2df8dee6" is_pre_election: true
I20260925 22:05:19.193773 7011 raft_consensus.cc:2454] T 9ec890372ccf4bfa975037a7f6e18399 P 874ee8b794ef4086a9601b6a2df8dee6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38876e68cdfa46158375464729ee36f4 in term 0.
I20260925 22:05:19.193742 7151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ec890372ccf4bfa975037a7f6e18399" candidate_uuid: "38876e68cdfa46158375464729ee36f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10148bd29f0445239c5b4e9793900f5f" is_pre_election: true
I20260925 22:05:19.193862 7151 raft_consensus.cc:2454] T 9ec890372ccf4bfa975037a7f6e18399 P 10148bd29f0445239c5b4e9793900f5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38876e68cdfa46158375464729ee36f4 in term 0.
I20260925 22:05:19.193971 7222 leader_election.cc:302] T 9ec890372ccf4bfa975037a7f6e18399 P 38876e68cdfa46158375464729ee36f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38876e68cdfa46158375464729ee36f4, 874ee8b794ef4086a9601b6a2df8dee6; no voters:
I20260925 22:05:19.194072 7430 raft_consensus.cc:2781] T 9ec890372ccf4bfa975037a7f6e18399 P 38876e68cdfa46158375464729ee36f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:05:19.194114 7430 raft_consensus.cc:482] T 9ec890372ccf4bfa975037a7f6e18399 P 38876e68cdfa46158375464729ee36f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:05:19.194135 7430 raft_consensus.cc:3037] T 9ec890372ccf4bfa975037a7f6e18399 P 38876e68cdfa46158375464729ee36f4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:19.194865 7430 raft_consensus.cc:504] T 9ec890372ccf4bfa975037a7f6e18399 P 38876e68cdfa46158375464729ee36f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:19.194998 7430 leader_election.cc:288] T 9ec890372ccf4bfa975037a7f6e18399 P 38876e68cdfa46158375464729ee36f4 [CANDIDATE]: Term 1 election: Requested vote from peers 874ee8b794ef4086a9601b6a2df8dee6 (127.5.0.65:36769), 10148bd29f0445239c5b4e9793900f5f (127.5.0.66:33273)
I20260925 22:05:19.195165 7151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ec890372ccf4bfa975037a7f6e18399" candidate_uuid: "38876e68cdfa46158375464729ee36f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10148bd29f0445239c5b4e9793900f5f"
I20260925 22:05:19.195160 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ec890372ccf4bfa975037a7f6e18399" candidate_uuid: "38876e68cdfa46158375464729ee36f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "874ee8b794ef4086a9601b6a2df8dee6"
I20260925 22:05:19.195251 7151 raft_consensus.cc:3037] T 9ec890372ccf4bfa975037a7f6e18399 P 10148bd29f0445239c5b4e9793900f5f [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:19.195248 7011 raft_consensus.cc:3037] T 9ec890372ccf4bfa975037a7f6e18399 P 874ee8b794ef4086a9601b6a2df8dee6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:19.196051 7011 raft_consensus.cc:2454] T 9ec890372ccf4bfa975037a7f6e18399 P 874ee8b794ef4086a9601b6a2df8dee6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38876e68cdfa46158375464729ee36f4 in term 1.
I20260925 22:05:19.196061 7151 raft_consensus.cc:2454] T 9ec890372ccf4bfa975037a7f6e18399 P 10148bd29f0445239c5b4e9793900f5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38876e68cdfa46158375464729ee36f4 in term 1.
I20260925 22:05:19.196234 7222 leader_election.cc:302] T 9ec890372ccf4bfa975037a7f6e18399 P 38876e68cdfa46158375464729ee36f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10148bd29f0445239c5b4e9793900f5f, 38876e68cdfa46158375464729ee36f4; no voters:
I20260925 22:05:19.196375 7430 raft_consensus.cc:2781] T 9ec890372ccf4bfa975037a7f6e18399 P 38876e68cdfa46158375464729ee36f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:19.196444 7430 raft_consensus.cc:686] T 9ec890372ccf4bfa975037a7f6e18399 P 38876e68cdfa46158375464729ee36f4 [term 1 LEADER]: Becoming Leader. State: Replica: 38876e68cdfa46158375464729ee36f4, State: Running, Role: LEADER
I20260925 22:05:19.196504 7430 consensus_queue.cc:237] T 9ec890372ccf4bfa975037a7f6e18399 P 38876e68cdfa46158375464729ee36f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } }
I20260925 22:05:19.197074 6853 catalog_manager.cc:5495] T 9ec890372ccf4bfa975037a7f6e18399 P 38876e68cdfa46158375464729ee36f4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 38876e68cdfa46158375464729ee36f4 (127.5.0.67). New cstate: current_term: 1 leader_uuid: "38876e68cdfa46158375464729ee36f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38876e68cdfa46158375464729ee36f4" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37045 } health_report { overall_health: HEALTHY } } }
I20260925 22:05:19.220216 7467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:19.217904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37302 (local address 127.5.0.67:37045)
0925 22:05:19.218064 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:19.218066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:19.218072 (+ 6us) server_negotiation.cc:407] Connection header received
0925 22:05:19.218089 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:19.218090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:19.218099 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:19.218148 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:05:19.218288 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:19.218815 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:19.219514 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:19.219709 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:19.219773 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:19.219868 (+ 95us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:05:19.219964 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:05:19.220019 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:19.220070 (+ 51us) server_negotiation.cc:299] Negotiation successful
0925 22:05:19.220089 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":111,"threads_started":1}
I20260925 22:05:19.222442 7151 raft_consensus.cc:1261] T 9ec890372ccf4bfa975037a7f6e18399 P 10148bd29f0445239c5b4e9793900f5f [term 1 FOLLOWER]: Refusing update from remote peer 38876e68cdfa46158375464729ee36f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:05:19.222491 7011 raft_consensus.cc:1261] T 9ec890372ccf4bfa975037a7f6e18399 P 874ee8b794ef4086a9601b6a2df8dee6 [term 1 FOLLOWER]: Refusing update from remote peer 38876e68cdfa46158375464729ee36f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:05:19.222705 7430 consensus_queue.cc:1035] T 9ec890372ccf4bfa975037a7f6e18399 P 38876e68cdfa46158375464729ee36f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "874ee8b794ef4086a9601b6a2df8dee6" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 36769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:05:19.222807 7433 consensus_queue.cc:1035] T 9ec890372ccf4bfa975037a7f6e18399 P 38876e68cdfa46158375464729ee36f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10148bd29f0445239c5b4e9793900f5f" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 22:05:19.227118 7422 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260925 22:05:20.746840 6852 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:05:20.746940 6852 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:05:20.747800 6852 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 196ba701dd2e4bb9a12bc02d704b73ce: Sending DeleteTablet for 3 replicas of tablet 9ec890372ccf4bfa975037a7f6e18399
I20260925 22:05:20.748117 7271 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ec890372ccf4bfa975037a7f6e18399 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:20 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:37280
I20260925 22:05:20.748162 7131 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ec890372ccf4bfa975037a7f6e18399 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:20 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:42444
I20260925 22:05:20.748309 6990 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ec890372ccf4bfa975037a7f6e18399 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:20 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:44022
I20260925 22:05:20.748323 7492 tablet_replica.cc:330] T 9ec890372ccf4bfa975037a7f6e18399 P 38876e68cdfa46158375464729ee36f4: stopping tablet replica
I20260925 22:05:20.748397 7492 raft_consensus.cc:2229] T 9ec890372ccf4bfa975037a7f6e18399 P 38876e68cdfa46158375464729ee36f4 [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:05:20.748392 7493 tablet_replica.cc:330] T 9ec890372ccf4bfa975037a7f6e18399 P 10148bd29f0445239c5b4e9793900f5f: stopping tablet replica
I20260925 22:05:20.748453 7493 raft_consensus.cc:2229] T 9ec890372ccf4bfa975037a7f6e18399 P 10148bd29f0445239c5b4e9793900f5f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:05:20.748463 7492 raft_consensus.cc:2258] T 9ec890372ccf4bfa975037a7f6e18399 P 38876e68cdfa46158375464729ee36f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:20.748490 7493 raft_consensus.cc:2258] T 9ec890372ccf4bfa975037a7f6e18399 P 10148bd29f0445239c5b4e9793900f5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:20.748589 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 6926
I20260925 22:05:20.749193 7492 ts_tablet_manager.cc:1905] T 9ec890372ccf4bfa975037a7f6e18399 P 38876e68cdfa46158375464729ee36f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:20.750464 7493 ts_tablet_manager.cc:1905] T 9ec890372ccf4bfa975037a7f6e18399 P 10148bd29f0445239c5b4e9793900f5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:20.751093 7492 ts_tablet_manager.cc:1918] T 9ec890372ccf4bfa975037a7f6e18399 P 38876e68cdfa46158375464729ee36f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 22:05:20.751158 7492 log.cc:1196] T 9ec890372ccf4bfa975037a7f6e18399 P 38876e68cdfa46158375464729ee36f4: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-2/wal/wals/9ec890372ccf4bfa975037a7f6e18399
I20260925 22:05:20.751427 7492 ts_tablet_manager.cc:1939] T 9ec890372ccf4bfa975037a7f6e18399 P 38876e68cdfa46158375464729ee36f4: Deleting consensus metadata
I20260925 22:05:20.751767 6839 catalog_manager.cc:4841] TS 38876e68cdfa46158375464729ee36f4 (127.5.0.67:37045): tablet 9ec890372ccf4bfa975037a7f6e18399 (table test-table [id=a2fbd548bdbb43538197c33380c0b968]) successfully deleted
I20260925 22:05:20.752256 7493 ts_tablet_manager.cc:1918] T 9ec890372ccf4bfa975037a7f6e18399 P 10148bd29f0445239c5b4e9793900f5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 22:05:20.752337 7493 log.cc:1196] T 9ec890372ccf4bfa975037a7f6e18399 P 10148bd29f0445239c5b4e9793900f5f: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373908987421-5121-0/minicluster-data/ts-1/wal/wals/9ec890372ccf4bfa975037a7f6e18399
I20260925 22:05:20.752543 7493 ts_tablet_manager.cc:1939] T 9ec890372ccf4bfa975037a7f6e18399 P 10148bd29f0445239c5b4e9793900f5f: Deleting consensus metadata
I20260925 22:05:20.752915 6838 catalog_manager.cc:4841] TS 10148bd29f0445239c5b4e9793900f5f (127.5.0.66:33273): tablet 9ec890372ccf4bfa975037a7f6e18399 (table test-table [id=a2fbd548bdbb43538197c33380c0b968]) successfully deleted
W20260925 22:05:20.755496 6838 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.0.65:36769 (error 108)
I20260925 22:05:20.755689 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 7066
I20260925 22:05:20.756395 7496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:20.755868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:36769 (local address 127.0.0.1:44044)
0925 22:05:20.756302 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:20.756349 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.65:36769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":78,"threads_started":1}
W20260925 22:05:20.756487 6838 catalog_manager.cc:4584] TS 874ee8b794ef4086a9601b6a2df8dee6 (127.5.0.65:36769): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9ec890372ccf4bfa975037a7f6e18399: Network error: Client connection negotiation failed: client connection to 127.5.0.65:36769: connect: Connection refused (error 111)
I20260925 22:05:20.761194 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 7206
I20260925 22:05:20.766729 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 6822
2026-09-25T22:05:20Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5505 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:05:20 dist-test-slave-7mqr krb5kdc[7499](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:05:20 dist-test-slave-7mqr krb5kdc[7499](info): set up 2 sockets
Sep 25 22:05:20 dist-test-slave-7mqr krb5kdc[7499](info): commencing operation
krb5kdc: starting...
W20260925 22:05:21.791785 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.004s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:05:21 dist-test-slave-7mqr krb5kdc[7499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:05:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:05:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.0.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
I20260925 22:05:21.918802 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:33977
--webserver_interface=127.5.0.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:41089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:33977
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:21.998529 7512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:21.998791 7512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:21.998831 7512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:22.000030 7512 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:05:22.000100 7512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:22.000124 7512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:22.000145 7512 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:05:22.000175 7512 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:05:22.001658 7512 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:41089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:33977
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:33977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:22.001931 7512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:22.002164 7512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:22.007045 7517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:22.007036 7520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:22.007036 7518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:22.007296 7512 server_base.cc:1034] running on GCE node
I20260925 22:05:22.007536 7512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:22.007795 7512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:22.008956 7512 hybrid_clock.cc:648] HybridClock initialized: now 1790373922008940 us; error 31 us; skew 500 ppm
Sep 25 22:05:22 dist-test-slave-7mqr krb5kdc[7499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373922, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:22.011562 7512 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:05:22.012069 7512 webserver.cc:460] Webserver started at http://127.5.0.126:44461/ using document root <none> and password file <none>
I20260925 22:05:22.012254 7512 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:22.012295 7512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:22.012380 7512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:22.013391 7512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "ba8ae383fb8d4573a77aba67f9099647"
format_stamp: "Formatted at 2026-09-25 22:05:22 on dist-test-slave-7mqr"
server_key: "ac16f48a39a7645766cad3e92c6aa676"
server_key_iv: "108e80f3160692b777a01cf5e7829bf3"
server_key_version: "encryptionkey@0"
I20260925 22:05:22.013688 7512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "ba8ae383fb8d4573a77aba67f9099647"
format_stamp: "Formatted at 2026-09-25 22:05:22 on dist-test-slave-7mqr"
server_key: "ac16f48a39a7645766cad3e92c6aa676"
server_key_iv: "108e80f3160692b777a01cf5e7829bf3"
server_key_version: "encryptionkey@0"
I20260925 22:05:22.014925 7512 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:05:22.015638 7527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:22.015826 7512 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:05:22.015959 7512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "ba8ae383fb8d4573a77aba67f9099647"
format_stamp: "Formatted at 2026-09-25 22:05:22 on dist-test-slave-7mqr"
server_key: "ac16f48a39a7645766cad3e92c6aa676"
server_key_iv: "108e80f3160692b777a01cf5e7829bf3"
server_key_version: "encryptionkey@0"
I20260925 22:05:22.016036 7512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:22.028571 7512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:22.030855 7512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:22.031014 7512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:22.035904 7512 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:33977
I20260925 22:05:22.035962 7579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:33977 every 8 connection(s)
I20260925 22:05:22.036366 7512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:05:22.037557 7580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:22.040800 7580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba8ae383fb8d4573a77aba67f9099647: Bootstrap starting.
I20260925 22:05:22.041427 7580 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba8ae383fb8d4573a77aba67f9099647: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:22.041785 7580 log.cc:824] T 00000000000000000000000000000000 P ba8ae383fb8d4573a77aba67f9099647: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:22.042785 7580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba8ae383fb8d4573a77aba67f9099647: No bootstrap required, opened a new log
I20260925 22:05:22.043468 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 7512
I20260925 22:05:22.043548 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:05:22.043735 5121 external_mini_cluster.cc:1428] Setting key 863cdea0138d4e7d4ce0f9c306408c5c
I20260925 22:05:22.045193 7580 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba8ae383fb8d4573a77aba67f9099647 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba8ae383fb8d4573a77aba67f9099647" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 33977 } }
I20260925 22:05:22.045405 7580 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba8ae383fb8d4573a77aba67f9099647 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:22.045441 7580 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba8ae383fb8d4573a77aba67f9099647 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba8ae383fb8d4573a77aba67f9099647, State: Initialized, Role: FOLLOWER
I20260925 22:05:22.045575 7580 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba8ae383fb8d4573a77aba67f9099647 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba8ae383fb8d4573a77aba67f9099647" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 33977 } }
I20260925 22:05:22.045665 7580 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba8ae383fb8d4573a77aba67f9099647 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:05:22.045698 7580 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba8ae383fb8d4573a77aba67f9099647 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:05:22.045745 7580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba8ae383fb8d4573a77aba67f9099647 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:22.046327 7580 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba8ae383fb8d4573a77aba67f9099647 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba8ae383fb8d4573a77aba67f9099647" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 33977 } }
I20260925 22:05:22.046435 7580 leader_election.cc:302] T 00000000000000000000000000000000 P ba8ae383fb8d4573a77aba67f9099647 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba8ae383fb8d4573a77aba67f9099647; no voters:
I20260925 22:05:22.046605 7580 leader_election.cc:288] T 00000000000000000000000000000000 P ba8ae383fb8d4573a77aba67f9099647 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:05:22.046657 7585 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba8ae383fb8d4573a77aba67f9099647 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:22.046810 7585 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba8ae383fb8d4573a77aba67f9099647 [term 1 LEADER]: Becoming Leader. State: Replica: ba8ae383fb8d4573a77aba67f9099647, State: Running, Role: LEADER
I20260925 22:05:22.046819 7580 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba8ae383fb8d4573a77aba67f9099647 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:05:22.046913 7585 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba8ae383fb8d4573a77aba67f9099647 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba8ae383fb8d4573a77aba67f9099647" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 33977 } }
I20260925 22:05:22.047331 7585 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba8ae383fb8d4573a77aba67f9099647 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba8ae383fb8d4573a77aba67f9099647. Latest consensus state: current_term: 1 leader_uuid: "ba8ae383fb8d4573a77aba67f9099647" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba8ae383fb8d4573a77aba67f9099647" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 33977 } } }
I20260925 22:05:22.047433 7585 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba8ae383fb8d4573a77aba67f9099647 [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:22.047340 7587 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba8ae383fb8d4573a77aba67f9099647 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba8ae383fb8d4573a77aba67f9099647" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba8ae383fb8d4573a77aba67f9099647" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 33977 } } }
I20260925 22:05:22.047571 7587 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba8ae383fb8d4573a77aba67f9099647 [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:22.047722 7592 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:05:22.048492 7592 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:05:22.050074 7592 catalog_manager.cc:1347] Generated new cluster ID: 374ebb78d0c64db3bdb05d7421495cf4
I20260925 22:05:22.050132 7592 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:05:22 dist-test-slave-7mqr krb5kdc[7499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:22.057113 7584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:22.044297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50942 (local address 127.5.0.126:33977)
0925 22:05:22.044492 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:22.044502 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:22.044538 (+ 36us) server_negotiation.cc:407] Connection header received
0925 22:05:22.045392 (+ 854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:22.045403 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:22.045468 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:22.045636 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:22.046319 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:22.048516 (+ 2197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:22.049543 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:22.050109 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:22.050510 (+ 401us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:22.052994 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:22.052997 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:22.052998 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:22.053010 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:22.055419 (+ 2409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:22.055816 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:22.055818 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:22.055819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:22.055938 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:22.056141 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:22.056142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:22.056142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:22.056415 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:22.056563 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:22.056680 (+ 117us) server_negotiation.cc:299] Negotiation successful
0925 22:05:22.056907 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":79,"threads_started":1}
I20260925 22:05:22.068564 7592 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:05:22.069080 7592 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:05:22.071625 7592 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba8ae383fb8d4573a77aba67f9099647: Generated new TSK 0
I20260925 22:05:22.071796 7592 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.0.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
I20260925 22:05:22.104565 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.65:0
--local_ip_for_outbound_sockets=127.5.0.65
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:33977
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:41089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:22.185142 7606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:22.185374 7606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:22.185412 7606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:22.186573 7606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:22.186640 7606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:22.186682 7606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.65
I20260925 22:05:22.188158 7606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:41089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:33977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.65
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:22.188410 7606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:22.188645 7606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:22.193403 7612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:22.193434 7614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:22.193374 7611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:22.193594 7606 server_base.cc:1034] running on GCE node
I20260925 22:05:22.193806 7606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:22.193991 7606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:22.195127 7606 hybrid_clock.cc:648] HybridClock initialized: now 1790373922195107 us; error 34 us; skew 500 ppm
Sep 25 22:05:22 dist-test-slave-7mqr krb5kdc[7499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373922, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:22.197604 7606 init.cc:377] Logged in from keytab as kudu/127.5.0.65@KRBTEST.COM (short username kudu)
I20260925 22:05:22.198009 7606 webserver.cc:460] Webserver started at http://127.5.0.65:33853/ using document root <none> and password file <none>
I20260925 22:05:22.198248 7606 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:22.198318 7606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:22.198464 7606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:22.199548 7606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-0/data/instance:
uuid: "0fea47eafb244043adf6c2afd7e50d8a"
format_stamp: "Formatted at 2026-09-25 22:05:22 on dist-test-slave-7mqr"
server_key: "07145943d3a51a8021ec565109c5fb53"
server_key_iv: "496916ab7c7dd943befad8790adae4e4"
server_key_version: "encryptionkey@0"
I20260925 22:05:22.199955 7606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance:
uuid: "0fea47eafb244043adf6c2afd7e50d8a"
format_stamp: "Formatted at 2026-09-25 22:05:22 on dist-test-slave-7mqr"
server_key: "07145943d3a51a8021ec565109c5fb53"
server_key_iv: "496916ab7c7dd943befad8790adae4e4"
server_key_version: "encryptionkey@0"
I20260925 22:05:22.201264 7606 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:05:22.201933 7621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:22.202105 7606 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:05:22.202200 7606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-0/wal
uuid: "0fea47eafb244043adf6c2afd7e50d8a"
format_stamp: "Formatted at 2026-09-25 22:05:22 on dist-test-slave-7mqr"
server_key: "07145943d3a51a8021ec565109c5fb53"
server_key_iv: "496916ab7c7dd943befad8790adae4e4"
server_key_version: "encryptionkey@0"
I20260925 22:05:22.202284 7606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:22.218425 7606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:22.220624 7606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:22.220789 7606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:22.221005 7606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:22.221315 7606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:22.221369 7606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:22.221410 7606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:22.221449 7606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:22.227032 7606 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.65:40719
I20260925 22:05:22.227123 7734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.65:40719 every 8 connection(s)
I20260925 22:05:22.227415 7606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
I20260925 22:05:22.229501 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 7606
I20260925 22:05:22.229595 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance
I20260925 22:05:22.229770 5121 external_mini_cluster.cc:1428] Setting key 2d3e7369f98f30aa0bc67c7b23efd179
Sep 25 22:05:22 dist-test-slave-7mqr krb5kdc[7499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373922, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:22.237493 7584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:22.228144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:54413 (local address 127.5.0.126:33977)
0925 22:05:22.228213 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:22.228215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:22.228416 (+ 201us) server_negotiation.cc:407] Connection header received
0925 22:05:22.228865 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:22.228867 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:22.228876 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:22.228931 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:22.230093 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:22.230763 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:22.231755 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:22.232042 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:22.232131 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:22.234491 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:22.234492 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:22.234492 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:22.234502 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:22.236157 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:22.236523 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:22.236524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:22.236525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:22.236590 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:22.236835 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:22.236836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:22.236837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:22.236979 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:22.237055 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:22.237291 (+ 236us) server_negotiation.cc:299] Negotiation successful
0925 22:05:22.237389 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 22:05:22.237526 7737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:22.228141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:33977 (local address 127.5.0.65:54413)
0925 22:05:22.228283 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:22.228322 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:22.228783 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:22.228981 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:22.228984 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:22.229706 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:22.229990 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:22.229996 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:22.230859 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:22.230859 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:22.231656 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:22.231659 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:22.232141 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:22.232142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:22.232276 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:22.232825 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:22.232835 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:22.234395 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:22.236231 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:22.236236 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:22.236238 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:22.236447 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:22.236664 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:22.236665 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:22.236665 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:22.236768 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:22.237048 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:22.237051 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:22.237161 (+ 110us) client_negotiation.cc:769] Sending connection context
0925 22:05:22.237246 (+ 85us) client_negotiation.cc:240] Negotiation successful
0925 22:05:22.237362 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":72,"threads_started":1}
I20260925 22:05:22.238098 7735 heartbeater.cc:347] Connected to a master server at 127.5.0.126:33977
I20260925 22:05:22.238206 7735 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:22.238375 7735 heartbeater.cc:511] Master 127.5.0.126:33977 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.5.0.66@KRBTEST.COM; defaulting to no policy
I20260925 22:05:22.238952 7544 ts_manager.cc:194] Registered new tserver with Master: 0fea47eafb244043adf6c2afd7e50d8a (127.5.0.65:40719)
I20260925 22:05:22.239554 7544 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.65@KRBTEST.COM'} at 127.5.0.65:54413
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
I20260925 22:05:22.258358 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.66:0
--local_ip_for_outbound_sockets=127.5.0.66
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:33977
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:41089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:22.338091 7740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:22.338297 7740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:22.338320 7740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:22.339525 7740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:22.339576 7740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:22.339603 7740 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.66
I20260925 22:05:22.341068 7740 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:41089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:33977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.66
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:22.341318 7740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:22.341553 7740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:22.346621 7748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:22.346626 7746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:22.346683 7740 server_base.cc:1034] running on GCE node
W20260925 22:05:22.346627 7745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:22.347121 7740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:22.347337 7740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:22.348467 7740 hybrid_clock.cc:648] HybridClock initialized: now 1790373922348458 us; error 31 us; skew 500 ppm
Sep 25 22:05:22 dist-test-slave-7mqr krb5kdc[7499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373922, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:22.350929 7740 init.cc:377] Logged in from keytab as kudu/127.5.0.66@KRBTEST.COM (short username kudu)
I20260925 22:05:22.351368 7740 webserver.cc:460] Webserver started at http://127.5.0.66:38617/ using document root <none> and password file <none>
I20260925 22:05:22.351588 7740 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:22.351632 7740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:22.351778 7740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:22.352914 7740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-1/data/instance:
uuid: "a9e3e16c43dc48feaf7e1ca43debc405"
format_stamp: "Formatted at 2026-09-25 22:05:22 on dist-test-slave-7mqr"
server_key: "e6e0886b9078a048c42aabdf79c8e4df"
server_key_iv: "e3c69e131d6b5d433fb1e662a08a21ec"
server_key_version: "encryptionkey@0"
I20260925 22:05:22.353274 7740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance:
uuid: "a9e3e16c43dc48feaf7e1ca43debc405"
format_stamp: "Formatted at 2026-09-25 22:05:22 on dist-test-slave-7mqr"
server_key: "e6e0886b9078a048c42aabdf79c8e4df"
server_key_iv: "e3c69e131d6b5d433fb1e662a08a21ec"
server_key_version: "encryptionkey@0"
I20260925 22:05:22.354535 7740 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:05:22.355209 7755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:22.355404 7740 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:05:22.355492 7740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-1/wal
uuid: "a9e3e16c43dc48feaf7e1ca43debc405"
format_stamp: "Formatted at 2026-09-25 22:05:22 on dist-test-slave-7mqr"
server_key: "e6e0886b9078a048c42aabdf79c8e4df"
server_key_iv: "e3c69e131d6b5d433fb1e662a08a21ec"
server_key_version: "encryptionkey@0"
I20260925 22:05:22.355564 7740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:22.370975 7740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:22.373189 7740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:22.373337 7740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:22.373582 7740 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:22.373917 7740 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:22.373972 7740 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:22.374015 7740 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:22.374044 7740 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:22.380033 7740 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.66:43161
I20260925 22:05:22.380481 7740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
I20260925 22:05:22.380525 7868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.66:43161 every 8 connection(s)
I20260925 22:05:22.382781 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 7740
I20260925 22:05:22.382886 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance
I20260925 22:05:22.383100 5121 external_mini_cluster.cc:1428] Setting key cccaa241ba528a62ee0081f553e2cef5
Sep 25 22:05:22 dist-test-slave-7mqr krb5kdc[7499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373922, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:22.391264 7584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:22.381226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:58315 (local address 127.5.0.126:33977)
0925 22:05:22.381272 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:22.381274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:22.381459 (+ 185us) server_negotiation.cc:407] Connection header received
0925 22:05:22.381927 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:22.381928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:22.381940 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:22.382023 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:22.383135 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:22.383844 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:22.385426 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:22.385768 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:22.385851 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:22.388299 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:22.388301 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:22.388301 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:22.388310 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:22.389773 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:22.390262 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:22.390263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:22.390263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:22.390327 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:22.390602 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:22.390603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:22.390604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:22.390727 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:22.390805 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:22.391088 (+ 283us) server_negotiation.cc:299] Negotiation successful
0925 22:05:22.391170 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 22:05:22.391474 7871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:22.381172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:33977 (local address 127.5.0.66:58315)
0925 22:05:22.381324 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:22.381365 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:22.381835 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:22.382076 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:22.382078 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:22.382703 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:22.383019 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:22.383026 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:22.383968 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:22.383969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:22.385322 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:22.385326 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:22.385858 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:22.385858 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:22.385961 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:22.386533 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:22.386543 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:22.388199 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:22.389883 (+ 1684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:22.389889 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:22.389891 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:22.390166 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:22.390397 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:22.390398 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:22.390398 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:22.390527 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:22.390796 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:22.390802 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:22.390947 (+ 145us) client_negotiation.cc:769] Sending connection context
0925 22:05:22.391065 (+ 118us) client_negotiation.cc:240] Negotiation successful
0925 22:05:22.391228 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":89,"threads_started":1}
I20260925 22:05:22.392000 7869 heartbeater.cc:347] Connected to a master server at 127.5.0.126:33977
WARNING: no policy specified for kudu/127.5.0.67@KRBTEST.COM; defaulting to no policy
I20260925 22:05:22.392109 7869 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:22.392289 7869 heartbeater.cc:511] Master 127.5.0.126:33977 requested a full tablet report, sending...
I20260925 22:05:22.392663 7544 ts_manager.cc:194] Registered new tserver with Master: a9e3e16c43dc48feaf7e1ca43debc405 (127.5.0.66:43161)
I20260925 22:05:22.393039 7544 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.66@KRBTEST.COM'} at 127.5.0.66:58315
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
I20260925 22:05:22.408923 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.67:0
--local_ip_for_outbound_sockets=127.5.0.67
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:33977
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:41089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:22.490018 7874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:22.490222 7874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:22.490245 7874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:22.491403 7874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:22.491446 7874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:22.491474 7874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.67
I20260925 22:05:22.492925 7874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:41089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:33977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.67
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:22.493168 7874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:22.493423 7874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:22.498281 7882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:22.498348 7874 server_base.cc:1034] running on GCE node
W20260925 22:05:22.498293 7880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:22.498296 7879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:22.498795 7874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:22.499053 7874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:22.500221 7874 hybrid_clock.cc:648] HybridClock initialized: now 1790373922500216 us; error 38 us; skew 500 ppm
Sep 25 22:05:22 dist-test-slave-7mqr krb5kdc[7499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373922, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:22.502605 7874 init.cc:377] Logged in from keytab as kudu/127.5.0.67@KRBTEST.COM (short username kudu)
I20260925 22:05:22.503062 7874 webserver.cc:460] Webserver started at http://127.5.0.67:40703/ using document root <none> and password file <none>
I20260925 22:05:22.503301 7874 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:22.503368 7874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:22.503515 7874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:22.504652 7874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-2/data/instance:
uuid: "f471d7884c9845cfade99238b98864f7"
format_stamp: "Formatted at 2026-09-25 22:05:22 on dist-test-slave-7mqr"
server_key: "69cee64c58bb1dafecd7efa314382eba"
server_key_iv: "368dabe10a50124aa07d5697d5271366"
server_key_version: "encryptionkey@0"
I20260925 22:05:22.505014 7874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance:
uuid: "f471d7884c9845cfade99238b98864f7"
format_stamp: "Formatted at 2026-09-25 22:05:22 on dist-test-slave-7mqr"
server_key: "69cee64c58bb1dafecd7efa314382eba"
server_key_iv: "368dabe10a50124aa07d5697d5271366"
server_key_version: "encryptionkey@0"
I20260925 22:05:22.506359 7874 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:05:22.507076 7889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:22.507253 7874 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:05:22.507354 7874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-2/wal
uuid: "f471d7884c9845cfade99238b98864f7"
format_stamp: "Formatted at 2026-09-25 22:05:22 on dist-test-slave-7mqr"
server_key: "69cee64c58bb1dafecd7efa314382eba"
server_key_iv: "368dabe10a50124aa07d5697d5271366"
server_key_version: "encryptionkey@0"
I20260925 22:05:22.507427 7874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:22.521873 7874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:22.523725 7874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:22.523941 7874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:22.524204 7874 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:22.524528 7874 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:22.524581 7874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:22.524622 7874 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:22.524652 7874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:22.530350 7874 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.67:38821
I20260925 22:05:22.530400 8002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.67:38821 every 8 connection(s)
I20260925 22:05:22.530766 7874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
I20260925 22:05:22.533764 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 7874
I20260925 22:05:22.533890 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance
I20260925 22:05:22.534135 5121 external_mini_cluster.cc:1428] Setting key 43e4cc6672913785c6fdc5893e120490
Sep 25 22:05:22 dist-test-slave-7mqr krb5kdc[7499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373922, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:22.540586 7584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:22.531531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:50543 (local address 127.5.0.126:33977)
0925 22:05:22.531581 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:22.531584 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:22.531827 (+ 243us) server_negotiation.cc:407] Connection header received
0925 22:05:22.532415 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:22.532416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:22.532428 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:22.532479 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:22.533406 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:22.533983 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:22.535352 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:22.535537 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:22.535595 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:22.537817 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:22.537819 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:22.537819 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:22.537828 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:22.539178 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:22.539597 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:22.539598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:22.539599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:22.539656 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:22.539938 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:22.539939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:22.539939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:22.540120 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:22.540196 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:22.540443 (+ 247us) server_negotiation.cc:299] Negotiation successful
0925 22:05:22.540517 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 22:05:22.540767 8005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:22.531514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:33977 (local address 127.5.0.67:50543)
0925 22:05:22.531669 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:22.531709 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:22.532317 (+ 608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:22.532536 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:22.532540 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:22.533123 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:22.533321 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:22.533326 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:22.534085 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:22.534086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:22.535273 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:22.535276 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:22.535611 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:22.535611 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:22.535709 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:22.536193 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:22.536201 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:22.537698 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:22.539281 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:22.539287 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:22.539289 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:22.539523 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:22.539721 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:22.539722 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:22.539722 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:22.539828 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:22.540188 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:22.540193 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:22.540322 (+ 129us) client_negotiation.cc:769] Sending connection context
0925 22:05:22.540412 (+ 90us) client_negotiation.cc:240] Negotiation successful
0925 22:05:22.540547 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":91,"threads_started":1}
I20260925 22:05:22.541314 8003 heartbeater.cc:347] Connected to a master server at 127.5.0.126:33977
I20260925 22:05:22.541430 8003 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:22.541623 8003 heartbeater.cc:511] Master 127.5.0.126:33977 requested a full tablet report, sending...
I20260925 22:05:22.542043 7544 ts_manager.cc:194] Registered new tserver with Master: f471d7884c9845cfade99238b98864f7 (127.5.0.67:38821)
I20260925 22:05:22.542456 7544 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.67@KRBTEST.COM'} at 127.5.0.67:50543
I20260925 22:05:22.545691 5121 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 22:05:22.557408 7584 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:22.556336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55792 (local address 127.5.0.126:33977)
0925 22:05:22.556416 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:22.556418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:22.556525 (+ 107us) server_negotiation.cc:407] Connection header received
0925 22:05:22.556613 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:22.556614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:22.556631 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:22.556694 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:22.557353 (+ 659us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55792: BlockingRecv error: recv got EOF from 127.0.0.1:55792 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 22:05:22.558425 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 7606
I20260925 22:05:22.562603 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 7740
I20260925 22:05:22.566706 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 7874
I20260925 22:05:22.570631 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 7512
2026-09-25T22:05:22Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1802 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:05:22 dist-test-slave-7mqr krb5kdc[8015](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:05:22 dist-test-slave-7mqr krb5kdc[8015](info): set up 2 sockets
Sep 25 22:05:22 dist-test-slave-7mqr krb5kdc[8015](info): commencing operation
krb5kdc: starting...
W20260925 22:05:23.603801 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:05:23 dist-test-slave-7mqr krb5kdc[8015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:05:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:05:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.0.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
I20260925 22:05:23.722862 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:37115
--webserver_interface=127.5.0.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:38429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:37115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:23.802963 8028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:23.803164 8028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:23.803185 8028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:23.804311 8028 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:05:23.804346 8028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:23.804360 8028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:23.804373 8028 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:05:23.804385 8028 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:05:23.805799 8028 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:38429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:37115
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:37115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:23.805987 8028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:23.806146 8028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:23.810777 8036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:23.810789 8033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:23.810791 8034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:23.811131 8028 server_base.cc:1034] running on GCE node
I20260925 22:05:23.811408 8028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:23.811684 8028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:23.812851 8028 hybrid_clock.cc:648] HybridClock initialized: now 1790373923812848 us; error 54 us; skew 500 ppm
Sep 25 22:05:23 dist-test-slave-7mqr krb5kdc[8015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373923, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:23.815225 8028 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:05:23.815662 8028 webserver.cc:460] Webserver started at http://127.5.0.126:34399/ using document root <none> and password file <none>
I20260925 22:05:23.815955 8028 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:23.816002 8028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:23.816138 8028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:23.817168 8028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "69ec84220cdb4be69ec240e8c5a1c53c"
format_stamp: "Formatted at 2026-09-25 22:05:23 on dist-test-slave-7mqr"
server_key: "08fcf060a4b1bff006265a9b7d330439"
server_key_iv: "8e13591cc9d54fc9fa6ffbbd0786a8bb"
server_key_version: "encryptionkey@0"
I20260925 22:05:23.817528 8028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "69ec84220cdb4be69ec240e8c5a1c53c"
format_stamp: "Formatted at 2026-09-25 22:05:23 on dist-test-slave-7mqr"
server_key: "08fcf060a4b1bff006265a9b7d330439"
server_key_iv: "8e13591cc9d54fc9fa6ffbbd0786a8bb"
server_key_version: "encryptionkey@0"
I20260925 22:05:23.818811 8028 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:05:23.819656 8043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:23.819850 8028 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:05:23.819969 8028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "69ec84220cdb4be69ec240e8c5a1c53c"
format_stamp: "Formatted at 2026-09-25 22:05:23 on dist-test-slave-7mqr"
server_key: "08fcf060a4b1bff006265a9b7d330439"
server_key_iv: "8e13591cc9d54fc9fa6ffbbd0786a8bb"
server_key_version: "encryptionkey@0"
I20260925 22:05:23.820024 8028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:23.829133 8028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:23.830814 8028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:23.830917 8028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:23.834533 8028 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:37115
I20260925 22:05:23.834625 8095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:37115 every 8 connection(s)
I20260925 22:05:23.834920 8028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:05:23.835775 8096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:23.837149 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 8028
I20260925 22:05:23.837215 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:05:23.837385 5121 external_mini_cluster.cc:1428] Setting key 22d6da4a8e9b95da2c0c70b157192e13
I20260925 22:05:23.838548 8096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69ec84220cdb4be69ec240e8c5a1c53c: Bootstrap starting.
I20260925 22:05:23.839226 8096 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69ec84220cdb4be69ec240e8c5a1c53c: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:23.839552 8096 log.cc:824] T 00000000000000000000000000000000 P 69ec84220cdb4be69ec240e8c5a1c53c: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:23.840337 8096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69ec84220cdb4be69ec240e8c5a1c53c: No bootstrap required, opened a new log
I20260925 22:05:23.842576 8096 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69ec84220cdb4be69ec240e8c5a1c53c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ec84220cdb4be69ec240e8c5a1c53c" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 37115 } }
I20260925 22:05:23.842779 8096 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69ec84220cdb4be69ec240e8c5a1c53c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:23.842823 8096 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69ec84220cdb4be69ec240e8c5a1c53c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69ec84220cdb4be69ec240e8c5a1c53c, State: Initialized, Role: FOLLOWER
I20260925 22:05:23.842962 8096 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69ec84220cdb4be69ec240e8c5a1c53c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ec84220cdb4be69ec240e8c5a1c53c" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 37115 } }
I20260925 22:05:23.843046 8096 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69ec84220cdb4be69ec240e8c5a1c53c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:05:23.843070 8096 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69ec84220cdb4be69ec240e8c5a1c53c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:05:23.843144 8096 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69ec84220cdb4be69ec240e8c5a1c53c [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:23.843670 8096 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69ec84220cdb4be69ec240e8c5a1c53c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ec84220cdb4be69ec240e8c5a1c53c" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 37115 } }
I20260925 22:05:23.843779 8096 leader_election.cc:302] T 00000000000000000000000000000000 P 69ec84220cdb4be69ec240e8c5a1c53c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69ec84220cdb4be69ec240e8c5a1c53c; no voters:
I20260925 22:05:23.843991 8096 leader_election.cc:288] T 00000000000000000000000000000000 P 69ec84220cdb4be69ec240e8c5a1c53c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:05:23.844053 8101 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69ec84220cdb4be69ec240e8c5a1c53c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:23.844195 8101 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69ec84220cdb4be69ec240e8c5a1c53c [term 1 LEADER]: Becoming Leader. State: Replica: 69ec84220cdb4be69ec240e8c5a1c53c, State: Running, Role: LEADER
I20260925 22:05:23.844257 8096 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69ec84220cdb4be69ec240e8c5a1c53c [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:05:23.844285 8101 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69ec84220cdb4be69ec240e8c5a1c53c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ec84220cdb4be69ec240e8c5a1c53c" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 37115 } }
Sep 25 22:05:23 dist-test-slave-7mqr krb5kdc[8015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:23.844632 8101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69ec84220cdb4be69ec240e8c5a1c53c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69ec84220cdb4be69ec240e8c5a1c53c. Latest consensus state: current_term: 1 leader_uuid: "69ec84220cdb4be69ec240e8c5a1c53c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ec84220cdb4be69ec240e8c5a1c53c" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 37115 } } }
I20260925 22:05:23.844726 8101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69ec84220cdb4be69ec240e8c5a1c53c [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:23.844965 8103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69ec84220cdb4be69ec240e8c5a1c53c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69ec84220cdb4be69ec240e8c5a1c53c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ec84220cdb4be69ec240e8c5a1c53c" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 37115 } } }
I20260925 22:05:23.845012 8107 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:05:23.845031 8103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69ec84220cdb4be69ec240e8c5a1c53c [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:23.845680 8107 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:05:23.849171 8099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:23.837881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34592 (local address 127.5.0.126:37115)
0925 22:05:23.838069 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:23.838078 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:23.838114 (+ 36us) server_negotiation.cc:407] Connection header received
0925 22:05:23.838660 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:23.838668 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:23.838728 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:23.838890 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:23.839473 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:23.840878 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:23.841605 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:23.842301 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:23.842417 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:23.845182 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:23.845186 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:23.845186 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:23.845197 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:23.847420 (+ 2223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:23.847779 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:23.847782 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:23.847783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:23.847945 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:23.848183 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:23.848184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:23.848185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:23.848498 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:23.848636 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:23.848822 (+ 186us) server_negotiation.cc:299] Negotiation successful
0925 22:05:23.848961 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":78,"threads_started":1}
I20260925 22:05:23.849184 8107 catalog_manager.cc:1347] Generated new cluster ID: f91edf7b39cb49c9b04b4cab1b94c406
I20260925 22:05:23.849239 8107 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:05:23.850028 5121 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:05:23.872368 8099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:23.867240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34596 (local address 127.5.0.126:37115)
0925 22:05:23.867303 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:23.867305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:23.867851 (+ 546us) server_negotiation.cc:407] Connection header received
0925 22:05:23.868112 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:23.868114 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:23.868126 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:23.868163 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:23.868855 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:23.869619 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:23.870630 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:23.870980 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:23.871074 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:23.871816 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:23.871818 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:23.871818 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:05:23.872094 (+ 276us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:23.872114 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:05:23.872121 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:23.872187 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:23.872246 (+ 59us) server_negotiation.cc:299] Negotiation successful
0925 22:05:23.872275 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 22:05:23.876855 8107 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:05:23.877384 8107 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:05:23.888973 8107 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69ec84220cdb4be69ec240e8c5a1c53c: Generated new TSK 0
I20260925 22:05:23.889216 8107 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:05:23.894603 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 8028
2026-09-25T22:05:23Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1322 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:05:23 dist-test-slave-7mqr krb5kdc[8129](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:05:23 dist-test-slave-7mqr krb5kdc[8129](info): set up 2 sockets
Sep 25 22:05:23 dist-test-slave-7mqr krb5kdc[8129](info): commencing operation
krb5kdc: starting...
W20260925 22:05:24.931728 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:05:25 dist-test-slave-7mqr krb5kdc[8129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:05:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:05:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.0.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
I20260925 22:05:25.046877 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:38151
--webserver_interface=127.5.0.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:41883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:38151
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:25.125583 8142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:25.125782 8142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:25.125802 8142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:25.126873 8142 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:05:25.126906 8142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:25.126920 8142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:25.126933 8142 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:05:25.126945 8142 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:05:25.128310 8142 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:41883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:38151
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:38151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:25.128484 8142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:25.128648 8142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:25.133311 8150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:25.133363 8142 server_base.cc:1034] running on GCE node
W20260925 22:05:25.133318 8148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:25.133468 8147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:25.133734 8142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:25.134037 8142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:25.135176 8142 hybrid_clock.cc:648] HybridClock initialized: now 1790373925135153 us; error 37 us; skew 500 ppm
Sep 25 22:05:25 dist-test-slave-7mqr krb5kdc[8129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373925, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:25.144264 8142 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:05:25.144695 8142 webserver.cc:460] Webserver started at http://127.5.0.126:37389/ using document root <none> and password file <none>
I20260925 22:05:25.144925 8142 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:25.144969 8142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:25.145133 8142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:25.146196 8142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "ad5c90e213eb48ccb2c4fe3e4ad9f942"
format_stamp: "Formatted at 2026-09-25 22:05:25 on dist-test-slave-7mqr"
server_key: "16d9a63acc2877042f93bfde1da7a500"
server_key_iv: "8c4d5a6d008e00bc6e1b01b5bcfa71ac"
server_key_version: "encryptionkey@0"
I20260925 22:05:25.146569 8142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "ad5c90e213eb48ccb2c4fe3e4ad9f942"
format_stamp: "Formatted at 2026-09-25 22:05:25 on dist-test-slave-7mqr"
server_key: "16d9a63acc2877042f93bfde1da7a500"
server_key_iv: "8c4d5a6d008e00bc6e1b01b5bcfa71ac"
server_key_version: "encryptionkey@0"
I20260925 22:05:25.147847 8142 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 22:05:25.148599 8157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:25.148799 8142 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:05:25.148888 8142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "ad5c90e213eb48ccb2c4fe3e4ad9f942"
format_stamp: "Formatted at 2026-09-25 22:05:25 on dist-test-slave-7mqr"
server_key: "16d9a63acc2877042f93bfde1da7a500"
server_key_iv: "8c4d5a6d008e00bc6e1b01b5bcfa71ac"
server_key_version: "encryptionkey@0"
I20260925 22:05:25.148960 8142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:25.161067 8142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:25.163010 8142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:25.163172 8142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:25.167045 8142 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:38151
I20260925 22:05:25.167079 8209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:38151 every 8 connection(s)
I20260925 22:05:25.167426 8142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:05:25.168434 8210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:25.170580 8210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad5c90e213eb48ccb2c4fe3e4ad9f942: Bootstrap starting.
I20260925 22:05:25.171087 8210 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad5c90e213eb48ccb2c4fe3e4ad9f942: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:25.171311 8210 log.cc:824] T 00000000000000000000000000000000 P ad5c90e213eb48ccb2c4fe3e4ad9f942: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:25.171923 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 8142
I20260925 22:05:25.171978 8210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad5c90e213eb48ccb2c4fe3e4ad9f942: No bootstrap required, opened a new log
I20260925 22:05:25.172053 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:05:25.172224 5121 external_mini_cluster.cc:1428] Setting key 3cf38c10e6025d2e05b995f4378d8f2a
I20260925 22:05:25.174147 8210 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad5c90e213eb48ccb2c4fe3e4ad9f942 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad5c90e213eb48ccb2c4fe3e4ad9f942" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 38151 } }
I20260925 22:05:25.174320 8210 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad5c90e213eb48ccb2c4fe3e4ad9f942 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:25.174356 8210 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad5c90e213eb48ccb2c4fe3e4ad9f942 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad5c90e213eb48ccb2c4fe3e4ad9f942, State: Initialized, Role: FOLLOWER
I20260925 22:05:25.174511 8210 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad5c90e213eb48ccb2c4fe3e4ad9f942 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad5c90e213eb48ccb2c4fe3e4ad9f942" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 38151 } }
I20260925 22:05:25.174576 8210 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad5c90e213eb48ccb2c4fe3e4ad9f942 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:05:25.174597 8210 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad5c90e213eb48ccb2c4fe3e4ad9f942 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:05:25.174669 8210 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad5c90e213eb48ccb2c4fe3e4ad9f942 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:25.175707 8210 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad5c90e213eb48ccb2c4fe3e4ad9f942 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad5c90e213eb48ccb2c4fe3e4ad9f942" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 38151 } }
I20260925 22:05:25.175809 8210 leader_election.cc:302] T 00000000000000000000000000000000 P ad5c90e213eb48ccb2c4fe3e4ad9f942 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad5c90e213eb48ccb2c4fe3e4ad9f942; no voters:
I20260925 22:05:25.175969 8210 leader_election.cc:288] T 00000000000000000000000000000000 P ad5c90e213eb48ccb2c4fe3e4ad9f942 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:05:25.176033 8215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad5c90e213eb48ccb2c4fe3e4ad9f942 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:25.176193 8215 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad5c90e213eb48ccb2c4fe3e4ad9f942 [term 1 LEADER]: Becoming Leader. State: Replica: ad5c90e213eb48ccb2c4fe3e4ad9f942, State: Running, Role: LEADER
I20260925 22:05:25.176224 8210 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad5c90e213eb48ccb2c4fe3e4ad9f942 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:05:25.176342 8215 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad5c90e213eb48ccb2c4fe3e4ad9f942 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad5c90e213eb48ccb2c4fe3e4ad9f942" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 38151 } }
I20260925 22:05:25.177420 8217 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad5c90e213eb48ccb2c4fe3e4ad9f942 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad5c90e213eb48ccb2c4fe3e4ad9f942. Latest consensus state: current_term: 1 leader_uuid: "ad5c90e213eb48ccb2c4fe3e4ad9f942" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad5c90e213eb48ccb2c4fe3e4ad9f942" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 38151 } } }
I20260925 22:05:25.177495 8217 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad5c90e213eb48ccb2c4fe3e4ad9f942 [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:25.177668 8230 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:05:25.177745 8216 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad5c90e213eb48ccb2c4fe3e4ad9f942 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad5c90e213eb48ccb2c4fe3e4ad9f942" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad5c90e213eb48ccb2c4fe3e4ad9f942" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 38151 } } }
I20260925 22:05:25.177913 8216 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad5c90e213eb48ccb2c4fe3e4ad9f942 [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:25.178308 8230 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:05:25.179916 8230 catalog_manager.cc:1347] Generated new cluster ID: 557a1baf34ee4f219cce71b39095bbe6
I20260925 22:05:25.179970 8230 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:05:25 dist-test-slave-7mqr krb5kdc[8129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:25.184684 8214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:25.172724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57878 (local address 127.5.0.126:38151)
0925 22:05:25.172909 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:25.172918 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:25.172952 (+ 34us) server_negotiation.cc:407] Connection header received
0925 22:05:25.174111 (+ 1159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:25.174119 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:25.174189 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:25.174341 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:25.175104 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:25.176598 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:25.177762 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:25.178405 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:25.178512 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:25.181292 (+ 2780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:25.181296 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:25.181296 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:25.181331 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:25.183090 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:25.183486 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:25.183489 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:25.183490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:25.183598 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:25.183839 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:25.183839 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:25.183840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:25.184125 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:25.184306 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:25.184375 (+ 69us) server_negotiation.cc:299] Negotiation successful
0925 22:05:25.184514 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":79,"threads_started":1}
I20260925 22:05:25.195991 8230 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:05:25.196529 8230 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:05:25.201515 8230 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad5c90e213eb48ccb2c4fe3e4ad9f942: Generated new TSK 0
I20260925 22:05:25.201716 8230 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.0.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
I20260925 22:05:25.232582 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.65:0
--local_ip_for_outbound_sockets=127.5.0.65
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:38151
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:41883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:25.320677 8236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:25.320925 8236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:25.320960 8236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:25.322134 8236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:25.322203 8236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:25.322247 8236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.65
I20260925 22:05:25.323698 8236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:41883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:38151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.65
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:25.324013 8236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:25.324249 8236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:25.329308 8244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:25.329280 8241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:25.329320 8236 server_base.cc:1034] running on GCE node
W20260925 22:05:25.329320 8242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:25.329737 8236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:25.329936 8236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:25.331068 8236 hybrid_clock.cc:648] HybridClock initialized: now 1790373925331055 us; error 28 us; skew 500 ppm
Sep 25 22:05:25 dist-test-slave-7mqr krb5kdc[8129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373925, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:25.333617 8236 init.cc:377] Logged in from keytab as kudu/127.5.0.65@KRBTEST.COM (short username kudu)
I20260925 22:05:25.334082 8236 webserver.cc:460] Webserver started at http://127.5.0.65:40717/ using document root <none> and password file <none>
I20260925 22:05:25.334280 8236 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:25.334321 8236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:25.334407 8236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:25.335421 8236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-0/data/instance:
uuid: "a27c38d21500438883ab7b1e7f59a563"
format_stamp: "Formatted at 2026-09-25 22:05:25 on dist-test-slave-7mqr"
server_key: "30ebddad7b7576150d0de71059175b5f"
server_key_iv: "b92374b785e57ea1f6440edbe92dc8ae"
server_key_version: "encryptionkey@0"
I20260925 22:05:25.335732 8236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance:
uuid: "a27c38d21500438883ab7b1e7f59a563"
format_stamp: "Formatted at 2026-09-25 22:05:25 on dist-test-slave-7mqr"
server_key: "30ebddad7b7576150d0de71059175b5f"
server_key_iv: "b92374b785e57ea1f6440edbe92dc8ae"
server_key_version: "encryptionkey@0"
I20260925 22:05:25.337046 8236 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 22:05:25.337713 8251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:25.337863 8236 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:25.337944 8236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-0/wal
uuid: "a27c38d21500438883ab7b1e7f59a563"
format_stamp: "Formatted at 2026-09-25 22:05:25 on dist-test-slave-7mqr"
server_key: "30ebddad7b7576150d0de71059175b5f"
server_key_iv: "b92374b785e57ea1f6440edbe92dc8ae"
server_key_version: "encryptionkey@0"
I20260925 22:05:25.337996 8236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:25.355665 8236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:25.357582 8236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:25.357700 8236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:25.357885 8236 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:25.358192 8236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:25.358225 8236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:25.358247 8236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:25.358294 8236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:25.363821 8236 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.65:33643
I20260925 22:05:25.363922 8364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.65:33643 every 8 connection(s)
I20260925 22:05:25.364243 8236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
I20260925 22:05:25.367763 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 8236
I20260925 22:05:25.367870 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance
I20260925 22:05:25.368112 5121 external_mini_cluster.cc:1428] Setting key 1ac1f787515f5c3f2727cd3a733d7175
Sep 25 22:05:25 dist-test-slave-7mqr krb5kdc[8129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373925, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:25.374831 8214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:25.364987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:36601 (local address 127.5.0.126:38151)
0925 22:05:25.365028 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:25.365030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:25.365240 (+ 210us) server_negotiation.cc:407] Connection header received
0925 22:05:25.365776 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:25.365778 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:25.365789 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:25.365842 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:25.366809 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:25.367324 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:25.368725 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:25.369048 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:25.369120 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:25.371466 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:25.371467 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:25.371468 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:25.371476 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:25.373275 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:25.373745 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:25.373746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:25.373747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:25.373814 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:25.374068 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:25.374069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:25.374070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:25.374227 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:25.374309 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:25.374607 (+ 298us) server_negotiation.cc:299] Negotiation successful
0925 22:05:25.374699 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 22:05:25.374861 8367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:25.364960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38151 (local address 127.5.0.65:36601)
0925 22:05:25.365126 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:25.365160 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:25.365673 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:25.365922 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:25.365926 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:25.366517 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:25.366706 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:25.366711 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:25.367430 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:25.367431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:25.368608 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:25.368611 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:25.369132 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:25.369132 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:25.369246 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:25.369794 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:25.369802 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:25.371374 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:25.373381 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:25.373387 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:25.373389 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:25.373654 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:25.373888 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:25.373889 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:25.373889 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:25.373999 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:25.374328 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:25.374334 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:25.374456 (+ 122us) client_negotiation.cc:769] Sending connection context
0925 22:05:25.374566 (+ 110us) client_negotiation.cc:240] Negotiation successful
0925 22:05:25.374692 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":78,"threads_started":1}
I20260925 22:05:25.375443 8365 heartbeater.cc:347] Connected to a master server at 127.5.0.126:38151
I20260925 22:05:25.375554 8365 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:25.375765 8365 heartbeater.cc:511] Master 127.5.0.126:38151 requested a full tablet report, sending...
I20260925 22:05:25.376374 8174 ts_manager.cc:194] Registered new tserver with Master: a27c38d21500438883ab7b1e7f59a563 (127.5.0.65:33643)
I20260925 22:05:25.377063 8174 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.65@KRBTEST.COM'} at 127.5.0.65:36601
WARNING: no policy specified for kudu/127.5.0.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
I20260925 22:05:25.393051 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.66:0
--local_ip_for_outbound_sockets=127.5.0.66
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:38151
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:41883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:25.472748 8370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:25.472944 8370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:25.472967 8370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:25.474123 8370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:25.474167 8370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:25.474193 8370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.66
I20260925 22:05:25.475653 8370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:41883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:38151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.66
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:25.475883 8370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:25.476150 8370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:25.481240 8378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:25.481220 8376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:25.481307 8375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:25.481416 8370 server_base.cc:1034] running on GCE node
I20260925 22:05:25.481695 8370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:25.481992 8370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:25.483151 8370 hybrid_clock.cc:648] HybridClock initialized: now 1790373925483141 us; error 30 us; skew 500 ppm
Sep 25 22:05:25 dist-test-slave-7mqr krb5kdc[8129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373925, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:25.485602 8370 init.cc:377] Logged in from keytab as kudu/127.5.0.66@KRBTEST.COM (short username kudu)
I20260925 22:05:25.486068 8370 webserver.cc:460] Webserver started at http://127.5.0.66:37523/ using document root <none> and password file <none>
I20260925 22:05:25.486320 8370 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:25.486390 8370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:25.486544 8370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:25.487574 8370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-1/data/instance:
uuid: "6a8aa8da6b5a4505903a1bc9c7d920eb"
format_stamp: "Formatted at 2026-09-25 22:05:25 on dist-test-slave-7mqr"
server_key: "b460fcd3e575577fb292dd350fa0a948"
server_key_iv: "abd404b15ace80a5a4e788aecd12301d"
server_key_version: "encryptionkey@0"
I20260925 22:05:25.488024 8370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance:
uuid: "6a8aa8da6b5a4505903a1bc9c7d920eb"
format_stamp: "Formatted at 2026-09-25 22:05:25 on dist-test-slave-7mqr"
server_key: "b460fcd3e575577fb292dd350fa0a948"
server_key_iv: "abd404b15ace80a5a4e788aecd12301d"
server_key_version: "encryptionkey@0"
I20260925 22:05:25.489320 8370 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:05:25.490033 8385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:25.490207 8370 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:05:25.490306 8370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-1/wal
uuid: "6a8aa8da6b5a4505903a1bc9c7d920eb"
format_stamp: "Formatted at 2026-09-25 22:05:25 on dist-test-slave-7mqr"
server_key: "b460fcd3e575577fb292dd350fa0a948"
server_key_iv: "abd404b15ace80a5a4e788aecd12301d"
server_key_version: "encryptionkey@0"
I20260925 22:05:25.490391 8370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:25.511409 8370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:25.513341 8370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:25.513525 8370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:25.513779 8370 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:25.514114 8370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:25.514168 8370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:25.514204 8370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:25.514226 8370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:25.520030 8370 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.66:45787
I20260925 22:05:25.520087 8498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.66:45787 every 8 connection(s)
I20260925 22:05:25.520432 8370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
Sep 25 22:05:25 dist-test-slave-7mqr krb5kdc[8129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373925, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:25.527827 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 8370
I20260925 22:05:25.527969 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance
I20260925 22:05:25.528220 5121 external_mini_cluster.cc:1428] Setting key 9e4ad6f9cf5f7d5598b8f71f258a8362
I20260925 22:05:25.529548 8214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:25.521192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:43777 (local address 127.5.0.126:38151)
0925 22:05:25.521255 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:25.521257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:25.521439 (+ 182us) server_negotiation.cc:407] Connection header received
0925 22:05:25.521920 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:25.521921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:25.521930 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:25.521992 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:25.522869 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:25.523347 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:25.524312 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:25.524491 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:25.524541 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:25.526587 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:25.526589 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:25.526589 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:25.526596 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:25.528110 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:25.528571 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:25.528572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:25.528573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:25.528639 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:25.528901 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:25.528902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:25.528902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:25.529018 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:25.529091 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:25.529358 (+ 267us) server_negotiation.cc:299] Negotiation successful
0925 22:05:25.529441 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 22:05:25.529736 8501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:25.521162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38151 (local address 127.5.0.66:43777)
0925 22:05:25.521315 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:25.521355 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:25.521841 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:25.522047 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:25.522051 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:25.522596 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:25.522797 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:25.522801 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:25.523421 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:25.523422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:25.524242 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:25.524244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:25.524559 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:25.524559 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:25.524653 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:25.525083 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:25.525091 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:25.526501 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:25.528199 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:25.528205 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:25.528207 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:25.528482 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:25.528705 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:25.528706 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:25.528706 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:25.528824 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:25.529081 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:25.529088 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:25.529227 (+ 139us) client_negotiation.cc:769] Sending connection context
0925 22:05:25.529337 (+ 110us) client_negotiation.cc:240] Negotiation successful
0925 22:05:25.529488 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
I20260925 22:05:25.530290 8499 heartbeater.cc:347] Connected to a master server at 127.5.0.126:38151
I20260925 22:05:25.530409 8499 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:25.530578 8499 heartbeater.cc:511] Master 127.5.0.126:38151 requested a full tablet report, sending...
I20260925 22:05:25.530968 8174 ts_manager.cc:194] Registered new tserver with Master: 6a8aa8da6b5a4505903a1bc9c7d920eb (127.5.0.66:45787)
I20260925 22:05:25.531450 8174 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.66@KRBTEST.COM'} at 127.5.0.66:43777
WARNING: no policy specified for kudu/127.5.0.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
I20260925 22:05:25.552529 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.67:0
--local_ip_for_outbound_sockets=127.5.0.67
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:38151
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:41883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:25.632217 8504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:25.632419 8504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:25.632443 8504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:25.633551 8504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:25.633587 8504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:25.633612 8504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.67
I20260925 22:05:25.634963 8504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:41883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:38151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.67
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:25.635149 8504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:25.635324 8504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:25.640167 8512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:25.640167 8509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:25.640235 8510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:25.640486 8504 server_base.cc:1034] running on GCE node
I20260925 22:05:25.640700 8504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:25.640962 8504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:25.642118 8504 hybrid_clock.cc:648] HybridClock initialized: now 1790373925642097 us; error 40 us; skew 500 ppm
Sep 25 22:05:25 dist-test-slave-7mqr krb5kdc[8129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373925, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:25.644656 8504 init.cc:377] Logged in from keytab as kudu/127.5.0.67@KRBTEST.COM (short username kudu)
I20260925 22:05:25.645116 8504 webserver.cc:460] Webserver started at http://127.5.0.67:33121/ using document root <none> and password file <none>
I20260925 22:05:25.645306 8504 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:25.645345 8504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:25.645433 8504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:25.646445 8504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-2/data/instance:
uuid: "478e38a1e79046e7ae2d146dbc075fb8"
format_stamp: "Formatted at 2026-09-25 22:05:25 on dist-test-slave-7mqr"
server_key: "ef9335a71d430a0a9b96a886b21356d6"
server_key_iv: "0ceb1d52777996314ca5580e0f35542b"
server_key_version: "encryptionkey@0"
I20260925 22:05:25.646762 8504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance:
uuid: "478e38a1e79046e7ae2d146dbc075fb8"
format_stamp: "Formatted at 2026-09-25 22:05:25 on dist-test-slave-7mqr"
server_key: "ef9335a71d430a0a9b96a886b21356d6"
server_key_iv: "0ceb1d52777996314ca5580e0f35542b"
server_key_version: "encryptionkey@0"
I20260925 22:05:25.648061 8504 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 22:05:25.648833 8519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:25.649012 8504 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:05:25.649082 8504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-2/wal
uuid: "478e38a1e79046e7ae2d146dbc075fb8"
format_stamp: "Formatted at 2026-09-25 22:05:25 on dist-test-slave-7mqr"
server_key: "ef9335a71d430a0a9b96a886b21356d6"
server_key_iv: "0ceb1d52777996314ca5580e0f35542b"
server_key_version: "encryptionkey@0"
I20260925 22:05:25.649128 8504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:25.674232 8504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:25.676381 8504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:25.676571 8504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:25.676819 8504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:25.677196 8504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:25.677261 8504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:25.677307 8504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:25.677335 8504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:25.683296 8504 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.67:44577
I20260925 22:05:25.683331 8632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.67:44577 every 8 connection(s)
I20260925 22:05:25.683670 8504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
I20260925 22:05:25.687147 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 8504
I20260925 22:05:25.687251 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance
I20260925 22:05:25.687436 5121 external_mini_cluster.cc:1428] Setting key c5b91f8d37692020b1bc82ac98397cfc
Sep 25 22:05:25 dist-test-slave-7mqr krb5kdc[8129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373925, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:25.693403 8214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:25.684470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:54319 (local address 127.5.0.126:38151)
0925 22:05:25.684550 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:25.684552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:25.684795 (+ 243us) server_negotiation.cc:407] Connection header received
0925 22:05:25.685332 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:25.685333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:25.685343 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:25.685396 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:25.686342 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:25.686848 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:25.688303 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:25.688580 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:25.688635 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:25.690781 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:25.690783 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:25.690783 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:25.690789 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:25.692171 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:25.692559 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:25.692559 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:25.692559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:25.692610 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:25.692825 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:25.692825 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:25.692825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:25.692926 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:25.693011 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:25.693274 (+ 263us) server_negotiation.cc:299] Negotiation successful
0925 22:05:25.693338 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 22:05:25.693606 8635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:25.684448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38151 (local address 127.5.0.67:54319)
0925 22:05:25.684628 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:25.684683 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:25.685243 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:25.685451 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:25.685454 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:25.686042 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:25.686246 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:25.686251 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:25.686923 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:25.686924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:25.688215 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:25.688218 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:25.688653 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:25.688654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:25.688765 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:25.689233 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:25.689241 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:25.690695 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:25.692252 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:25.692257 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:25.692258 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:25.692487 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:25.692663 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:25.692664 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:25.692664 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:25.692772 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:25.693022 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:25.693026 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:25.693158 (+ 132us) client_negotiation.cc:769] Sending connection context
0925 22:05:25.693244 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 22:05:25.693365 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":73,"threads_started":1}
I20260925 22:05:25.694190 8633 heartbeater.cc:347] Connected to a master server at 127.5.0.126:38151
I20260925 22:05:25.694283 8633 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:25.694435 8633 heartbeater.cc:511] Master 127.5.0.126:38151 requested a full tablet report, sending...
I20260925 22:05:25.694775 8174 ts_manager.cc:194] Registered new tserver with Master: 478e38a1e79046e7ae2d146dbc075fb8 (127.5.0.67:44577)
I20260925 22:05:25.695106 8174 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.67@KRBTEST.COM'} at 127.5.0.67:54319
I20260925 22:05:25.699137 5121 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 22:05:25 dist-test-slave-7mqr krb5kdc[8129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373925, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 25 22:05:25 dist-test-slave-7mqr krb5kdc[8129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373925, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:25.721062 8214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:25.714117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57888 (local address 127.5.0.126:38151)
0925 22:05:25.714200 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:25.714203 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:25.714290 (+ 87us) server_negotiation.cc:407] Connection header received
0925 22:05:25.714397 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:25.714398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:25.714409 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:25.714482 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:25.715087 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:25.715584 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:25.716427 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:25.716624 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:25.716709 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:25.718920 (+ 2211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:25.718922 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:25.718922 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:25.718929 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:25.720176 (+ 1247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:25.720471 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:25.720472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:25.720472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:25.720522 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:25.720667 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:25.720667 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:25.720667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:25.720792 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:25.720862 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:25.720940 (+ 78us) server_negotiation.cc:299] Negotiation successful
0925 22:05:25.720996 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260925 22:05:25.721324 8174 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:57888
I20260925 22:05:25.721844 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 8236
I20260925 22:05:25.726930 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 8370
I20260925 22:05:25.731009 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 8504
I20260925 22:05:25.734994 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 8142
2026-09-25T22:05:25Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1842 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:05:25 dist-test-slave-7mqr krb5kdc[8645](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:05:25 dist-test-slave-7mqr krb5kdc[8645](info): set up 2 sockets
Sep 25 22:05:25 dist-test-slave-7mqr krb5kdc[8645](info): commencing operation
krb5kdc: starting...
W20260925 22:05:26.827864 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:05:26 dist-test-slave-7mqr krb5kdc[8645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:05:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:05:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.0.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
I20260925 22:05:26.946151 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:43811
--webserver_interface=127.5.0.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:36557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:43811
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:27.026554 8658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:27.026763 8658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:27.026786 8658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:27.027992 8658 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:05:27.028033 8658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:27.028047 8658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:27.028095 8658 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:05:27.028128 8658 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:05:27.029644 8658 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:36557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:43811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:43811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:27.029906 8658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:27.030180 8658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:27.035027 8663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:27.035007 8666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:27.035007 8664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:27.035034 8658 server_base.cc:1034] running on GCE node
I20260925 22:05:27.035465 8658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:27.035815 8658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:27.037034 8658 hybrid_clock.cc:648] HybridClock initialized: now 1790373927037039 us; error 49 us; skew 500 ppm
Sep 25 22:05:27 dist-test-slave-7mqr krb5kdc[8645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373927, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:27.039516 8658 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:05:27.040066 8658 webserver.cc:460] Webserver started at http://127.5.0.126:46751/ using document root <none> and password file <none>
I20260925 22:05:27.040323 8658 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:27.040393 8658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:27.040537 8658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:27.041558 8658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "20e1279d7aa84222bc88f006e50ed676"
format_stamp: "Formatted at 2026-09-25 22:05:27 on dist-test-slave-7mqr"
server_key: "0d8cd800ab44343f41bd5d074eeb4378"
server_key_iv: "751ecd74e4acce78c5dc85a86ec953bb"
server_key_version: "encryptionkey@0"
I20260925 22:05:27.041852 8658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "20e1279d7aa84222bc88f006e50ed676"
format_stamp: "Formatted at 2026-09-25 22:05:27 on dist-test-slave-7mqr"
server_key: "0d8cd800ab44343f41bd5d074eeb4378"
server_key_iv: "751ecd74e4acce78c5dc85a86ec953bb"
server_key_version: "encryptionkey@0"
I20260925 22:05:27.043243 8658 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:05:27.043952 8673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:27.044126 8658 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:05:27.044216 8658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "20e1279d7aa84222bc88f006e50ed676"
format_stamp: "Formatted at 2026-09-25 22:05:27 on dist-test-slave-7mqr"
server_key: "0d8cd800ab44343f41bd5d074eeb4378"
server_key_iv: "751ecd74e4acce78c5dc85a86ec953bb"
server_key_version: "encryptionkey@0"
I20260925 22:05:27.044274 8658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:27.058888 8658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:27.060819 8658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:27.060930 8658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:27.064735 8658 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:43811
I20260925 22:05:27.064785 8725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:43811 every 8 connection(s)
I20260925 22:05:27.065269 8658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:05:27.065953 8726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:27.068176 8726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20e1279d7aa84222bc88f006e50ed676: Bootstrap starting.
I20260925 22:05:27.068702 8726 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20e1279d7aa84222bc88f006e50ed676: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:27.068928 8726 log.cc:824] T 00000000000000000000000000000000 P 20e1279d7aa84222bc88f006e50ed676: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:27.069557 8726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20e1279d7aa84222bc88f006e50ed676: No bootstrap required, opened a new log
I20260925 22:05:27.070988 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 8658
I20260925 22:05:27.071096 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:05:27.071281 5121 external_mini_cluster.cc:1428] Setting key 27a6f22a816e1e156b97772d64c16952
I20260925 22:05:27.071341 8726 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20e1279d7aa84222bc88f006e50ed676 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20e1279d7aa84222bc88f006e50ed676" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43811 } }
I20260925 22:05:27.071513 8726 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20e1279d7aa84222bc88f006e50ed676 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:27.071539 8726 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20e1279d7aa84222bc88f006e50ed676 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20e1279d7aa84222bc88f006e50ed676, State: Initialized, Role: FOLLOWER
I20260925 22:05:27.071707 8726 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20e1279d7aa84222bc88f006e50ed676 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20e1279d7aa84222bc88f006e50ed676" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43811 } }
I20260925 22:05:27.071877 8726 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20e1279d7aa84222bc88f006e50ed676 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:05:27.071961 8726 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20e1279d7aa84222bc88f006e50ed676 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:05:27.072034 8726 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20e1279d7aa84222bc88f006e50ed676 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:27.072715 8726 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20e1279d7aa84222bc88f006e50ed676 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20e1279d7aa84222bc88f006e50ed676" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43811 } }
I20260925 22:05:27.072837 8726 leader_election.cc:302] T 00000000000000000000000000000000 P 20e1279d7aa84222bc88f006e50ed676 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20e1279d7aa84222bc88f006e50ed676; no voters:
I20260925 22:05:27.073035 8726 leader_election.cc:288] T 00000000000000000000000000000000 P 20e1279d7aa84222bc88f006e50ed676 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:05:27.073067 8731 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20e1279d7aa84222bc88f006e50ed676 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:27.073195 8731 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20e1279d7aa84222bc88f006e50ed676 [term 1 LEADER]: Becoming Leader. State: Replica: 20e1279d7aa84222bc88f006e50ed676, State: Running, Role: LEADER
I20260925 22:05:27.073276 8731 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20e1279d7aa84222bc88f006e50ed676 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20e1279d7aa84222bc88f006e50ed676" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43811 } }
I20260925 22:05:27.073329 8726 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20e1279d7aa84222bc88f006e50ed676 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:05:27.073596 8731 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20e1279d7aa84222bc88f006e50ed676 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "20e1279d7aa84222bc88f006e50ed676" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20e1279d7aa84222bc88f006e50ed676" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43811 } } }
I20260925 22:05:27.073665 8731 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20e1279d7aa84222bc88f006e50ed676 [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:27.073827 8731 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20e1279d7aa84222bc88f006e50ed676 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20e1279d7aa84222bc88f006e50ed676. Latest consensus state: current_term: 1 leader_uuid: "20e1279d7aa84222bc88f006e50ed676" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20e1279d7aa84222bc88f006e50ed676" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43811 } } }
I20260925 22:05:27.073937 8737 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:05:27.073961 8731 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20e1279d7aa84222bc88f006e50ed676 [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:27.074488 8737 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:05:27.076196 8737 catalog_manager.cc:1347] Generated new cluster ID: 610943bc2f4d4a4e88d9b53332f88132
I20260925 22:05:27.076254 8737 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:05:27 dist-test-slave-7mqr krb5kdc[8645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:27.083297 8730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:27.071828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38500 (local address 127.5.0.126:43811)
0925 22:05:27.072473 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:27.072481 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:27.072507 (+ 26us) server_negotiation.cc:407] Connection header received
0925 22:05:27.072605 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:27.072610 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:27.072652 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:27.072803 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:27.073590 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:27.075083 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:27.076195 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:27.076805 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:27.076923 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:27.079676 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:27.079680 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:27.079680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:27.079692 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:27.081669 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:27.082075 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:27.082078 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:27.082078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:27.082189 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:27.082403 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:27.082404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:27.082404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:27.082690 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:27.082849 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:27.082957 (+ 108us) server_negotiation.cc:299] Negotiation successful
0925 22:05:27.083087 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":556,"threads_started":1}
I20260925 22:05:27.097169 8737 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:05:27.097935 8737 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:05:27.100960 8737 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 20e1279d7aa84222bc88f006e50ed676: Generated new TSK 0
I20260925 22:05:27.101178 8737 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.0.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
I20260925 22:05:27.135123 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.65:0
--local_ip_for_outbound_sockets=127.5.0.65
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:43811
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:36557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:27.233777 8752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:27.233996 8752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:27.234035 8752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:27.235378 8752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:27.235462 8752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:27.235512 8752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.65
I20260925 22:05:27.237282 8752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:36557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:43811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.65
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:27.237546 8752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:27.237774 8752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:27.242740 8758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:27.242740 8760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:27.243057 8752 server_base.cc:1034] running on GCE node
W20260925 22:05:27.242779 8757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:27.243305 8752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:27.243518 8752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:27.244661 8752 hybrid_clock.cc:648] HybridClock initialized: now 1790373927244647 us; error 28 us; skew 500 ppm
Sep 25 22:05:27 dist-test-slave-7mqr krb5kdc[8645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373927, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:27.247136 8752 init.cc:377] Logged in from keytab as kudu/127.5.0.65@KRBTEST.COM (short username kudu)
I20260925 22:05:27.247607 8752 webserver.cc:460] Webserver started at http://127.5.0.65:45707/ using document root <none> and password file <none>
I20260925 22:05:27.247960 8752 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:27.248013 8752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:27.248160 8752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:27.249207 8752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-0/data/instance:
uuid: "ce6f14c2303241e3ba3d59e52845a736"
format_stamp: "Formatted at 2026-09-25 22:05:27 on dist-test-slave-7mqr"
server_key: "0f562e065923fc5f1c6d325f4b85c7bc"
server_key_iv: "7bac9b7b96244cfeeb95dcf977088ab6"
server_key_version: "encryptionkey@0"
I20260925 22:05:27.249573 8752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance:
uuid: "ce6f14c2303241e3ba3d59e52845a736"
format_stamp: "Formatted at 2026-09-25 22:05:27 on dist-test-slave-7mqr"
server_key: "0f562e065923fc5f1c6d325f4b85c7bc"
server_key_iv: "7bac9b7b96244cfeeb95dcf977088ab6"
server_key_version: "encryptionkey@0"
I20260925 22:05:27.250928 8752 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:05:27.251649 8767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:27.251835 8752 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:05:27.251959 8752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-0/wal
uuid: "ce6f14c2303241e3ba3d59e52845a736"
format_stamp: "Formatted at 2026-09-25 22:05:27 on dist-test-slave-7mqr"
server_key: "0f562e065923fc5f1c6d325f4b85c7bc"
server_key_iv: "7bac9b7b96244cfeeb95dcf977088ab6"
server_key_version: "encryptionkey@0"
I20260925 22:05:27.252049 8752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:27.259955 8752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:27.261705 8752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:27.261883 8752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:27.262115 8752 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:27.262432 8752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:27.262486 8752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:27.262521 8752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:27.262548 8752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:27.268239 8752 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.65:39493
I20260925 22:05:27.268283 8880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.65:39493 every 8 connection(s)
I20260925 22:05:27.268590 8752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
I20260925 22:05:27.270186 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 8752
I20260925 22:05:27.270292 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance
I20260925 22:05:27.270530 5121 external_mini_cluster.cc:1428] Setting key 257c042c7309d6753647187561afed96
Sep 25 22:05:27 dist-test-slave-7mqr krb5kdc[8645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373927, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:27.278627 8730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:27.269284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:49669 (local address 127.5.0.126:43811)
0925 22:05:27.269362 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:27.269364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:27.269530 (+ 166us) server_negotiation.cc:407] Connection header received
0925 22:05:27.270496 (+ 966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:27.270497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:27.270515 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:27.270587 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:27.271207 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:27.271711 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:27.273161 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:27.273467 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:27.273534 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:27.275761 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:27.275763 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:27.275763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:27.275772 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:27.277269 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:27.277674 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:27.277676 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:27.277676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:27.277745 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:27.277989 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:27.277990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:27.277991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:27.278109 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:27.278183 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:27.278449 (+ 266us) server_negotiation.cc:299] Negotiation successful
0925 22:05:27.278537 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 22:05:27.278805 8883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:27.269236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:43811 (local address 127.5.0.65:49669)
0925 22:05:27.269413 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:27.269447 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:27.270393 (+ 946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:27.270669 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:27.270675 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:27.270921 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:27.271102 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:27.271105 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:27.271819 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:27.271821 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:27.273086 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:27.273089 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:27.273541 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:27.273542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:27.273702 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:27.274346 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:27.274355 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:27.275664 (+ 1309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:27.277368 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:27.277375 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:27.277377 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:27.277596 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:27.277811 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:27.277811 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:27.277812 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:27.277922 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:27.278175 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:27.278181 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:27.278321 (+ 140us) client_negotiation.cc:769] Sending connection context
0925 22:05:27.278428 (+ 107us) client_negotiation.cc:240] Negotiation successful
0925 22:05:27.278579 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":91,"threads_started":1}
I20260925 22:05:27.279280 8881 heartbeater.cc:347] Connected to a master server at 127.5.0.126:43811
I20260925 22:05:27.279356 8881 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.5.0.66@KRBTEST.COM; defaulting to no policy
I20260925 22:05:27.279541 8881 heartbeater.cc:511] Master 127.5.0.126:43811 requested a full tablet report, sending...
I20260925 22:05:27.280114 8690 ts_manager.cc:194] Registered new tserver with Master: ce6f14c2303241e3ba3d59e52845a736 (127.5.0.65:39493)
I20260925 22:05:27.280800 8690 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.65@KRBTEST.COM'} at 127.5.0.65:49669
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
I20260925 22:05:27.296154 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.66:0
--local_ip_for_outbound_sockets=127.5.0.66
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:43811
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:36557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:27.376017 8886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:27.376219 8886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:27.376243 8886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:27.377373 8886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:27.377414 8886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:27.377467 8886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.66
I20260925 22:05:27.378925 8886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:36557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:43811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.66
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:27.379165 8886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:27.379395 8886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:27.384178 8894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:27.384259 8891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:27.384267 8892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:27.384356 8886 server_base.cc:1034] running on GCE node
I20260925 22:05:27.384583 8886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:27.384781 8886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:27.385923 8886 hybrid_clock.cc:648] HybridClock initialized: now 1790373927385903 us; error 37 us; skew 500 ppm
Sep 25 22:05:27 dist-test-slave-7mqr krb5kdc[8645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373927, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:27.388476 8886 init.cc:377] Logged in from keytab as kudu/127.5.0.66@KRBTEST.COM (short username kudu)
I20260925 22:05:27.388927 8886 webserver.cc:460] Webserver started at http://127.5.0.66:42257/ using document root <none> and password file <none>
I20260925 22:05:27.389153 8886 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:27.389221 8886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:27.389369 8886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:27.390353 8886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-1/data/instance:
uuid: "786be1bf0f3d4907b871eccdfabc8d0f"
format_stamp: "Formatted at 2026-09-25 22:05:27 on dist-test-slave-7mqr"
server_key: "ffcc552c6f2ac66d3399643a535368b1"
server_key_iv: "6bd8ba339c6606871a07d641316fae58"
server_key_version: "encryptionkey@0"
I20260925 22:05:27.390715 8886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance:
uuid: "786be1bf0f3d4907b871eccdfabc8d0f"
format_stamp: "Formatted at 2026-09-25 22:05:27 on dist-test-slave-7mqr"
server_key: "ffcc552c6f2ac66d3399643a535368b1"
server_key_iv: "6bd8ba339c6606871a07d641316fae58"
server_key_version: "encryptionkey@0"
I20260925 22:05:27.392015 8886 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 22:05:27.392724 8901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:27.392895 8886 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:05:27.392988 8886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-1/wal
uuid: "786be1bf0f3d4907b871eccdfabc8d0f"
format_stamp: "Formatted at 2026-09-25 22:05:27 on dist-test-slave-7mqr"
server_key: "ffcc552c6f2ac66d3399643a535368b1"
server_key_iv: "6bd8ba339c6606871a07d641316fae58"
server_key_version: "encryptionkey@0"
I20260925 22:05:27.393072 8886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:27.406486 8886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:27.408814 8886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:27.408975 8886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:27.409200 8886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:27.409500 8886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:27.409552 8886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:27.409586 8886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:27.409612 8886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:27.415129 8886 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.66:38001
I20260925 22:05:27.415172 9014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.66:38001 every 8 connection(s)
I20260925 22:05:27.415485 8886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
I20260925 22:05:27.420643 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 8886
I20260925 22:05:27.420742 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance
I20260925 22:05:27.420915 5121 external_mini_cluster.cc:1428] Setting key d5e67f064500ec4719b34e107979429b
Sep 25 22:05:27 dist-test-slave-7mqr krb5kdc[8645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373927, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:27.424959 8730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:27.416276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:34907 (local address 127.5.0.126:43811)
0925 22:05:27.416346 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:27.416348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:27.416506 (+ 158us) server_negotiation.cc:407] Connection header received
0925 22:05:27.417486 (+ 980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:27.417488 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:27.417497 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:27.417558 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:27.418136 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:27.418637 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:27.419575 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:27.419761 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:27.419809 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:27.422030 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:27.422032 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:27.422032 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:27.422040 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:27.423469 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:27.424013 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:27.424014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:27.424015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:27.424065 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:27.424342 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:27.424342 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:27.424343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:27.424443 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:27.424492 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:27.424827 (+ 335us) server_negotiation.cc:299] Negotiation successful
0925 22:05:27.424896 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 22:05:27.425271 9017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:27.416228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:43811 (local address 127.5.0.66:34907)
0925 22:05:27.416384 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:27.416424 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:27.417391 (+ 967us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:27.417627 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:27.417630 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:27.417878 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:27.418062 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:27.418067 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:27.418716 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:27.418716 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:27.419503 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:27.419504 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:27.419828 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:27.419829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:27.419937 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:27.420375 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:27.420383 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:27.421940 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:27.423554 (+ 1614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:27.423562 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:27.423564 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:27.423930 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:27.424133 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:27.424134 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:27.424134 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:27.424277 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:27.424507 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:27.424512 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:27.424681 (+ 169us) client_negotiation.cc:769] Sending connection context
0925 22:05:27.424808 (+ 127us) client_negotiation.cc:240] Negotiation successful
0925 22:05:27.424986 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
I20260925 22:05:27.425808 9015 heartbeater.cc:347] Connected to a master server at 127.5.0.126:43811
I20260925 22:05:27.425896 9015 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:27.426079 9015 heartbeater.cc:511] Master 127.5.0.126:43811 requested a full tablet report, sending...
I20260925 22:05:27.426533 8690 ts_manager.cc:194] Registered new tserver with Master: 786be1bf0f3d4907b871eccdfabc8d0f (127.5.0.66:38001)
I20260925 22:05:27.426913 8690 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.66@KRBTEST.COM'} at 127.5.0.66:34907
WARNING: no policy specified for kudu/127.5.0.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
I20260925 22:05:27.446448 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.67:0
--local_ip_for_outbound_sockets=127.5.0.67
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:43811
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:36557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:27.526283 9020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:27.526551 9020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:27.526588 9020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:27.527762 9020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:27.527832 9020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:27.527875 9020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.67
I20260925 22:05:27.529413 9020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:36557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:43811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.67
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:27.529677 9020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:27.529914 9020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:27.535543 9028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:27.535543 9026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:27.535588 9025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:27.535926 9020 server_base.cc:1034] running on GCE node
I20260925 22:05:27.536180 9020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:27.536470 9020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:27.537637 9020 hybrid_clock.cc:648] HybridClock initialized: now 1790373927537638 us; error 36 us; skew 500 ppm
Sep 25 22:05:27 dist-test-slave-7mqr krb5kdc[8645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373927, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:27.540158 9020 init.cc:377] Logged in from keytab as kudu/127.5.0.67@KRBTEST.COM (short username kudu)
I20260925 22:05:27.540696 9020 webserver.cc:460] Webserver started at http://127.5.0.67:44385/ using document root <none> and password file <none>
I20260925 22:05:27.540946 9020 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:27.541016 9020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:27.541155 9020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:27.542248 9020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-2/data/instance:
uuid: "04364e6274a44c3fb9a620bdbf8929fd"
format_stamp: "Formatted at 2026-09-25 22:05:27 on dist-test-slave-7mqr"
server_key: "51c1559af328db35264a88b1bf52bf6c"
server_key_iv: "395c0622dafae8eec7092e4ac03f0162"
server_key_version: "encryptionkey@0"
I20260925 22:05:27.542634 9020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance:
uuid: "04364e6274a44c3fb9a620bdbf8929fd"
format_stamp: "Formatted at 2026-09-25 22:05:27 on dist-test-slave-7mqr"
server_key: "51c1559af328db35264a88b1bf52bf6c"
server_key_iv: "395c0622dafae8eec7092e4ac03f0162"
server_key_version: "encryptionkey@0"
I20260925 22:05:27.544023 9020 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:05:27.544802 9035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:27.544989 9020 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:05:27.545064 9020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-2/wal
uuid: "04364e6274a44c3fb9a620bdbf8929fd"
format_stamp: "Formatted at 2026-09-25 22:05:27 on dist-test-slave-7mqr"
server_key: "51c1559af328db35264a88b1bf52bf6c"
server_key_iv: "395c0622dafae8eec7092e4ac03f0162"
server_key_version: "encryptionkey@0"
I20260925 22:05:27.545154 9020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:27.560952 9020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:27.562861 9020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:27.563017 9020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:27.563258 9020 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:27.563551 9020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:27.563583 9020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:27.563606 9020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:27.563648 9020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:27.569312 9020 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.67:43523
I20260925 22:05:27.569375 9148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.67:43523 every 8 connection(s)
I20260925 22:05:27.569680 9020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
I20260925 22:05:27.570943 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 9020
I20260925 22:05:27.571067 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance
I20260925 22:05:27.571275 5121 external_mini_cluster.cc:1428] Setting key 7beb7fb0d902f11f0c60a29b95789546
Sep 25 22:05:27 dist-test-slave-7mqr krb5kdc[8645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373927, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:27.579020 8730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:27.570437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:40997 (local address 127.5.0.126:43811)
0925 22:05:27.570508 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:27.570511 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:27.570713 (+ 202us) server_negotiation.cc:407] Connection header received
0925 22:05:27.571856 (+ 1143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:27.571857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:27.571867 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:27.571970 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:27.572724 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:27.573301 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:27.574385 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:27.574583 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:27.574638 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:27.576752 (+ 2114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:27.576753 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:27.576753 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:27.576760 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:27.577920 (+ 1160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:27.578277 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:27.578278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:27.578278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:27.578323 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:27.578525 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:27.578526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:27.578526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:27.578619 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:27.578666 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:27.578900 (+ 234us) server_negotiation.cc:299] Negotiation successful
0925 22:05:27.578956 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 22:05:27.579246 9151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:27.570426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:43811 (local address 127.5.0.67:40997)
0925 22:05:27.570591 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:27.570627 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:27.571761 (+ 1134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:27.572039 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:27.572042 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:27.572445 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:27.572650 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:27.572656 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:27.573375 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:27.573376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:27.574311 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:27.574313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:27.574656 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:27.574657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:27.574757 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:27.575295 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:27.575303 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:27.576663 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:27.577987 (+ 1324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:27.577992 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:27.577994 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:27.578202 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:27.578378 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:27.578379 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:27.578379 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:27.578472 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:27.578671 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:27.578674 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:27.578793 (+ 119us) client_negotiation.cc:769] Sending connection context
0925 22:05:27.578873 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 22:05:27.578983 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":83,"threads_started":1}
I20260925 22:05:27.579766 9149 heartbeater.cc:347] Connected to a master server at 127.5.0.126:43811
I20260925 22:05:27.579872 9149 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:27.580062 9149 heartbeater.cc:511] Master 127.5.0.126:43811 requested a full tablet report, sending...
I20260925 22:05:27.580453 8690 ts_manager.cc:194] Registered new tserver with Master: 04364e6274a44c3fb9a620bdbf8929fd (127.5.0.67:43523)
I20260925 22:05:27.580796 8690 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.67@KRBTEST.COM'} at 127.5.0.67:40997
I20260925 22:05:27.583267 5121 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 22:05:27 dist-test-slave-7mqr krb5kdc[8645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 22:05:27 dist-test-slave-7mqr krb5kdc[8645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.65@KRBTEST.COM
I20260925 22:05:27.605571 9156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:27.598329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47612 (local address 127.5.0.65:39493)
0925 22:05:27.598486 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:27.598496 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:27.598507 (+ 11us) server_negotiation.cc:407] Connection header received
0925 22:05:27.598532 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:27.598534 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:27.598547 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:27.598622 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:27.599172 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:27.599869 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:27.600634 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:27.600982 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:27.601033 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:27.603215 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:27.603222 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:27.603222 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:27.603229 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:27.604630 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:27.604941 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:27.604944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:27.604944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:27.604994 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:27.605154 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:27.605154 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:27.605155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:27.605272 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:27.605378 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:27.605433 (+ 55us) server_negotiation.cc:299] Negotiation successful
0925 22:05:27.605492 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":88,"threads_started":1}
Sep 25 22:05:27 dist-test-slave-7mqr krb5kdc[8645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:27.613451 8730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:27.606818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38514 (local address 127.5.0.126:43811)
0925 22:05:27.606890 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:27.606892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:27.606939 (+ 47us) server_negotiation.cc:407] Connection header received
0925 22:05:27.607001 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:27.607002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:27.607012 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:27.607068 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:27.607508 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:27.608058 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:27.608741 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:27.608947 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:27.609030 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:27.611066 (+ 2036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:27.611067 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:27.611068 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:27.611077 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:27.612322 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:27.612663 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:27.612665 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:27.612665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:27.612723 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:27.612917 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:27.612918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:27.612918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:27.613069 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:27.613169 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:27.613273 (+ 104us) server_negotiation.cc:299] Negotiation successful
0925 22:05:27.613349 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260925 22:05:27.613690 8690 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38514
I20260925 22:05:27.614001 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 8752
I20260925 22:05:27.618227 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 8886
I20260925 22:05:27.622572 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 9020
I20260925 22:05:27.626644 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 8658
2026-09-25T22:05:27Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1891 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:05:27 dist-test-slave-7mqr krb5kdc[9161](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:05:27 dist-test-slave-7mqr krb5kdc[9161](info): set up 2 sockets
Sep 25 22:05:27 dist-test-slave-7mqr krb5kdc[9161](info): commencing operation
krb5kdc: starting...
W20260925 22:05:28.659873 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:05:28 dist-test-slave-7mqr krb5kdc[9161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373928, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:05:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:05:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.0.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
I20260925 22:05:28.774434 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:39749
--webserver_interface=127.5.0.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:42577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:39749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:28.854378 9174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:28.854619 9174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:28.854656 9174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:28.855818 9174 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:05:28.855890 9174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:28.855952 9174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:28.855978 9174 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:05:28.856005 9174 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:05:28.857515 9174 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:42577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:39749
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:39749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:28.857785 9174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:28.858012 9174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:28.862744 9180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:28.862772 9174 server_base.cc:1034] running on GCE node
W20260925 22:05:28.862746 9182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:28.862802 9179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:28.863219 9174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:28.863547 9174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:28.864714 9174 hybrid_clock.cc:648] HybridClock initialized: now 1790373928864694 us; error 45 us; skew 500 ppm
Sep 25 22:05:28 dist-test-slave-7mqr krb5kdc[9161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373928, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:28.867105 9174 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:05:28.867475 9174 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:28.867543 9174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:28.867686 9174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:28.868794 9174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "df7f04264e474c6d9f99d2a181e709a8"
format_stamp: "Formatted at 2026-09-25 22:05:28 on dist-test-slave-7mqr"
server_key: "879220bbd9648b7b2230f95423d7a0dd"
server_key_iv: "4b8f252591f2e327f64b80425636040d"
server_key_version: "encryptionkey@0"
I20260925 22:05:28.869155 9174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "df7f04264e474c6d9f99d2a181e709a8"
format_stamp: "Formatted at 2026-09-25 22:05:28 on dist-test-slave-7mqr"
server_key: "879220bbd9648b7b2230f95423d7a0dd"
server_key_iv: "4b8f252591f2e327f64b80425636040d"
server_key_version: "encryptionkey@0"
I20260925 22:05:28.870443 9174 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:05:28.871161 9188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:28.871347 9174 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:05:28.871450 9174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "df7f04264e474c6d9f99d2a181e709a8"
format_stamp: "Formatted at 2026-09-25 22:05:28 on dist-test-slave-7mqr"
server_key: "879220bbd9648b7b2230f95423d7a0dd"
server_key_iv: "4b8f252591f2e327f64b80425636040d"
server_key_version: "encryptionkey@0"
I20260925 22:05:28.871541 9174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:28.881276 9174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:28.883023 9174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:28.883195 9174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:28.887606 9174 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:39749
I20260925 22:05:28.887642 9240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:39749 every 8 connection(s)
I20260925 22:05:28.888010 9174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:05:28.888876 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 9174
I20260925 22:05:28.888974 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:05:28.889159 5121 external_mini_cluster.cc:1428] Setting key adb80a91f34ea151081ad37e09fd8af7
I20260925 22:05:28.889215 9241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:28.892724 9241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df7f04264e474c6d9f99d2a181e709a8: Bootstrap starting.
I20260925 22:05:28.893375 9241 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df7f04264e474c6d9f99d2a181e709a8: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:28.893666 9241 log.cc:824] T 00000000000000000000000000000000 P df7f04264e474c6d9f99d2a181e709a8: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:28.894421 9241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df7f04264e474c6d9f99d2a181e709a8: No bootstrap required, opened a new log
Sep 25 22:05:28 dist-test-slave-7mqr krb5kdc[9161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373928, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:28.896905 9241 raft_consensus.cc:348] T 00000000000000000000000000000000 P df7f04264e474c6d9f99d2a181e709a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df7f04264e474c6d9f99d2a181e709a8" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 39749 } }
I20260925 22:05:28.897059 9241 raft_consensus.cc:374] T 00000000000000000000000000000000 P df7f04264e474c6d9f99d2a181e709a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:28.897083 9241 raft_consensus.cc:729] T 00000000000000000000000000000000 P df7f04264e474c6d9f99d2a181e709a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df7f04264e474c6d9f99d2a181e709a8, State: Initialized, Role: FOLLOWER
I20260925 22:05:28.897228 9241 consensus_queue.cc:260] T 00000000000000000000000000000000 P df7f04264e474c6d9f99d2a181e709a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df7f04264e474c6d9f99d2a181e709a8" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 39749 } }
I20260925 22:05:28.897300 9241 raft_consensus.cc:388] T 00000000000000000000000000000000 P df7f04264e474c6d9f99d2a181e709a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:05:28.897322 9241 raft_consensus.cc:482] T 00000000000000000000000000000000 P df7f04264e474c6d9f99d2a181e709a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:05:28.897393 9241 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df7f04264e474c6d9f99d2a181e709a8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:28.897898 9241 raft_consensus.cc:504] T 00000000000000000000000000000000 P df7f04264e474c6d9f99d2a181e709a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df7f04264e474c6d9f99d2a181e709a8" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 39749 } }
I20260925 22:05:28.898007 9241 leader_election.cc:302] T 00000000000000000000000000000000 P df7f04264e474c6d9f99d2a181e709a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df7f04264e474c6d9f99d2a181e709a8; no voters:
I20260925 22:05:28.898180 9241 leader_election.cc:288] T 00000000000000000000000000000000 P df7f04264e474c6d9f99d2a181e709a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:05:28.898236 9246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df7f04264e474c6d9f99d2a181e709a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:28.898411 9246 raft_consensus.cc:686] T 00000000000000000000000000000000 P df7f04264e474c6d9f99d2a181e709a8 [term 1 LEADER]: Becoming Leader. State: Replica: df7f04264e474c6d9f99d2a181e709a8, State: Running, Role: LEADER
I20260925 22:05:28.898433 9241 sys_catalog.cc:564] T 00000000000000000000000000000000 P df7f04264e474c6d9f99d2a181e709a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:05:28.898502 9246 consensus_queue.cc:237] T 00000000000000000000000000000000 P df7f04264e474c6d9f99d2a181e709a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df7f04264e474c6d9f99d2a181e709a8" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 39749 } }
I20260925 22:05:28.898849 9247 sys_catalog.cc:455] T 00000000000000000000000000000000 P df7f04264e474c6d9f99d2a181e709a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df7f04264e474c6d9f99d2a181e709a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df7f04264e474c6d9f99d2a181e709a8" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 39749 } } }
I20260925 22:05:28.898988 9247 sys_catalog.cc:458] T 00000000000000000000000000000000 P df7f04264e474c6d9f99d2a181e709a8 [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:28.899173 9247 sys_catalog.cc:455] T 00000000000000000000000000000000 P df7f04264e474c6d9f99d2a181e709a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df7f04264e474c6d9f99d2a181e709a8. Latest consensus state: current_term: 1 leader_uuid: "df7f04264e474c6d9f99d2a181e709a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df7f04264e474c6d9f99d2a181e709a8" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 39749 } } }
I20260925 22:05:28.899292 9247 sys_catalog.cc:458] T 00000000000000000000000000000000 P df7f04264e474c6d9f99d2a181e709a8 [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:28.900027 9255 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:05:28.900537 9255 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:05:28.900899 9244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:28.889765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51554 (local address 127.5.0.126:39749)
0925 22:05:28.889947 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:28.889957 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:28.889992 (+ 35us) server_negotiation.cc:407] Connection header received
0925 22:05:28.890454 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:28.890463 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:28.890530 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:28.890684 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:28.891263 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:28.892618 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:28.893363 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:28.893983 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:28.894105 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:28.896929 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:28.896933 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:28.896933 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:28.896943 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:28.898997 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:28.899447 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:28.899450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:28.899450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:28.899564 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:28.899817 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:28.899817 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:28.899818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:28.900159 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:28.900427 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:28.900546 (+ 119us) server_negotiation.cc:299] Negotiation successful
0925 22:05:28.900679 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
I20260925 22:05:28.902275 9255 catalog_manager.cc:1347] Generated new cluster ID: 94da24a50d18467eb59889300a588fbc
I20260925 22:05:28.902330 9255 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:05:28.908664 9255 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:05:28.909188 9255 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:05:28.914324 9255 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df7f04264e474c6d9f99d2a181e709a8: Generated new TSK 0
I20260925 22:05:28.914515 9255 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.0.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
I20260925 22:05:28.944655 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.65:0
--local_ip_for_outbound_sockets=127.5.0.65
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:39749
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:42577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:29.024560 9267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:29.024760 9267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:29.024782 9267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:29.025858 9267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:29.025892 9267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:29.025918 9267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.65
I20260925 22:05:29.027356 9267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:42577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:39749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.65
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:29.027539 9267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:29.027719 9267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:29.032804 9273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:29.032938 9272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:29.032985 9267 server_base.cc:1034] running on GCE node
W20260925 22:05:29.032776 9275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:29.033249 9267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:29.033455 9267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:29.034622 9267 hybrid_clock.cc:648] HybridClock initialized: now 1790373929034617 us; error 36 us; skew 500 ppm
Sep 25 22:05:29 dist-test-slave-7mqr krb5kdc[9161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373929, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:29.037257 9267 init.cc:377] Logged in from keytab as kudu/127.5.0.65@KRBTEST.COM (short username kudu)
I20260925 22:05:29.037662 9267 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:29.037719 9267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:29.037811 9267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:29.038884 9267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-0/data/instance:
uuid: "aaa24921abf74771b5f13044b8c6d68b"
format_stamp: "Formatted at 2026-09-25 22:05:29 on dist-test-slave-7mqr"
server_key: "463cb8747a6ea897a07410f91a11d44c"
server_key_iv: "8205a4a9a5adc3056c9bff9507598531"
server_key_version: "encryptionkey@0"
I20260925 22:05:29.039289 9267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance:
uuid: "aaa24921abf74771b5f13044b8c6d68b"
format_stamp: "Formatted at 2026-09-25 22:05:29 on dist-test-slave-7mqr"
server_key: "463cb8747a6ea897a07410f91a11d44c"
server_key_iv: "8205a4a9a5adc3056c9bff9507598531"
server_key_version: "encryptionkey@0"
I20260925 22:05:29.040782 9267 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:05:29.041533 9281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:29.041734 9267 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:29.041815 9267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-0/wal
uuid: "aaa24921abf74771b5f13044b8c6d68b"
format_stamp: "Formatted at 2026-09-25 22:05:29 on dist-test-slave-7mqr"
server_key: "463cb8747a6ea897a07410f91a11d44c"
server_key_iv: "8205a4a9a5adc3056c9bff9507598531"
server_key_version: "encryptionkey@0"
I20260925 22:05:29.041873 9267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:29.057056 9267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:29.059434 9267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:29.059546 9267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:29.059744 9267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:29.060066 9267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:29.060102 9267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:29.060124 9267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:29.060137 9267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:29.065814 9267 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.65:45031
I20260925 22:05:29.065872 9394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.65:45031 every 8 connection(s)
I20260925 22:05:29.066166 9267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
I20260925 22:05:29.069662 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 9267
I20260925 22:05:29.069779 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance
I20260925 22:05:29.069960 5121 external_mini_cluster.cc:1428] Setting key 6c16925e504482bd8a5e3ad3303bfe66
Sep 25 22:05:29 dist-test-slave-7mqr krb5kdc[9161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373929, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:29.076023 9244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:29.066924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:55325 (local address 127.5.0.126:39749)
0925 22:05:29.066999 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:29.067001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:29.067158 (+ 157us) server_negotiation.cc:407] Connection header received
0925 22:05:29.067614 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:29.067615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:29.067625 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:29.067675 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:29.068645 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.069140 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.070242 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.070515 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.070576 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:29.072819 (+ 2243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:29.072820 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:29.072820 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:29.072828 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:29.074531 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:29.075003 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:29.075004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:29.075004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:29.075068 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:29.075338 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:29.075339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:29.075339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:29.075466 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:29.075545 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:29.075818 (+ 273us) server_negotiation.cc:299] Negotiation successful
0925 22:05:29.075930 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 22:05:29.076220 9397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:29.066858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:39749 (local address 127.5.0.65:55325)
0925 22:05:29.067043 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:29.067079 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:29.067536 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:29.067746 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:29.067749 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:29.068069 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:29.068564 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:29.068570 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.069221 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.069222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:29.070148 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:29.070150 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.070598 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.070599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:29.070736 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:29.071286 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:29.071293 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:29.072704 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:29.074658 (+ 1954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:29.074665 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:29.074667 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:29.074921 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:29.075154 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:29.075155 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:29.075155 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:29.075268 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:29.075535 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:29.075541 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:29.075683 (+ 142us) client_negotiation.cc:769] Sending connection context
0925 22:05:29.075799 (+ 116us) client_negotiation.cc:240] Negotiation successful
0925 22:05:29.075974 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":97,"threads_started":1}
I20260925 22:05:29.076735 9395 heartbeater.cc:347] Connected to a master server at 127.5.0.126:39749
I20260925 22:05:29.076843 9395 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:29.077000 9395 heartbeater.cc:511] Master 127.5.0.126:39749 requested a full tablet report, sending...
I20260925 22:05:29.077495 9205 ts_manager.cc:194] Registered new tserver with Master: aaa24921abf74771b5f13044b8c6d68b (127.5.0.65:45031)
WARNING: no policy specified for kudu/127.5.0.66@KRBTEST.COM; defaulting to no policy
I20260925 22:05:29.078172 9205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.65@KRBTEST.COM'} at 127.5.0.65:55325
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
I20260925 22:05:29.094080 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.66:0
--local_ip_for_outbound_sockets=127.5.0.66
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:39749
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:42577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:29.175964 9400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:29.176175 9400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:29.176199 9400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:29.177366 9400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:29.177410 9400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:29.177438 9400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.66
I20260925 22:05:29.178929 9400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:42577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:39749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.66
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:29.179175 9400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:29.179411 9400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:29.184258 9406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:29.184253 9405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:29.184253 9408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:29.184464 9400 server_base.cc:1034] running on GCE node
I20260925 22:05:29.184814 9400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:29.185092 9400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:29.186259 9400 hybrid_clock.cc:648] HybridClock initialized: now 1790373929186246 us; error 43 us; skew 500 ppm
Sep 25 22:05:29 dist-test-slave-7mqr krb5kdc[9161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373929, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:29.188810 9400 init.cc:377] Logged in from keytab as kudu/127.5.0.66@KRBTEST.COM (short username kudu)
I20260925 22:05:29.189229 9400 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:29.189301 9400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:29.189451 9400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:29.190619 9400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-1/data/instance:
uuid: "6bb97950538540339e7eccb20d86de3a"
format_stamp: "Formatted at 2026-09-25 22:05:29 on dist-test-slave-7mqr"
server_key: "e108ab47788f429b6342c20a5bcaea7f"
server_key_iv: "0cf01e3e61be3c5526e347c9e49ba9f0"
server_key_version: "encryptionkey@0"
I20260925 22:05:29.190987 9400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance:
uuid: "6bb97950538540339e7eccb20d86de3a"
format_stamp: "Formatted at 2026-09-25 22:05:29 on dist-test-slave-7mqr"
server_key: "e108ab47788f429b6342c20a5bcaea7f"
server_key_iv: "0cf01e3e61be3c5526e347c9e49ba9f0"
server_key_version: "encryptionkey@0"
I20260925 22:05:29.192325 9400 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:05:29.193112 9414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:29.193284 9400 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:05:29.193387 9400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-1/wal
uuid: "6bb97950538540339e7eccb20d86de3a"
format_stamp: "Formatted at 2026-09-25 22:05:29 on dist-test-slave-7mqr"
server_key: "e108ab47788f429b6342c20a5bcaea7f"
server_key_iv: "0cf01e3e61be3c5526e347c9e49ba9f0"
server_key_version: "encryptionkey@0"
I20260925 22:05:29.193459 9400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:29.208964 9400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:29.210871 9400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:29.211045 9400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:29.211309 9400 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:29.211635 9400 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:29.211688 9400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:29.211730 9400 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:29.211771 9400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:29.217783 9400 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.66:39001
I20260925 22:05:29.217820 9527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.66:39001 every 8 connection(s)
I20260925 22:05:29.218091 9400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
I20260925 22:05:29.219099 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 9400
I20260925 22:05:29.219199 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance
I20260925 22:05:29.219408 5121 external_mini_cluster.cc:1428] Setting key cb22816d52a568b14968e82071e0c055
Sep 25 22:05:29 dist-test-slave-7mqr krb5kdc[9161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373929, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.0.67@KRBTEST.COM; defaulting to no policy
I20260925 22:05:29.228245 9244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:29.218769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:58467 (local address 127.5.0.126:39749)
0925 22:05:29.218851 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:29.218853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:29.219072 (+ 219us) server_negotiation.cc:407] Connection header received
0925 22:05:29.219749 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:29.219750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:29.219762 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:29.219820 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:29.220813 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.221469 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.222465 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.222805 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.222881 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:29.225081 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:29.225082 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:29.225083 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:29.225090 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:29.226730 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:29.227124 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:29.227125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:29.227126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:29.227188 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:29.227426 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:29.227427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:29.227428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:29.227561 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:29.227634 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:29.228009 (+ 375us) server_negotiation.cc:299] Negotiation successful
0925 22:05:29.228180 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 22:05:29.228286 9530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:29.218730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:39749 (local address 127.5.0.66:58467)
0925 22:05:29.218929 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:29.218968 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:29.219633 (+ 665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:29.219922 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:29.219927 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:29.220204 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:29.220708 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:29.220714 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.221570 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.221571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:29.222366 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:29.222368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.222890 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.222891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:29.222992 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:29.223552 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:29.223561 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:29.224980 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:29.226815 (+ 1835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:29.226820 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:29.226822 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:29.227042 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:29.227264 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:29.227265 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:29.227265 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:29.227360 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:29.227650 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:29.227655 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:29.227803 (+ 148us) client_negotiation.cc:769] Sending connection context
0925 22:05:29.227975 (+ 172us) client_negotiation.cc:240] Negotiation successful
0925 22:05:29.228105 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":121,"threads_started":1}
I20260925 22:05:29.228962 9528 heartbeater.cc:347] Connected to a master server at 127.5.0.126:39749
I20260925 22:05:29.229053 9528 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:29.229218 9528 heartbeater.cc:511] Master 127.5.0.126:39749 requested a full tablet report, sending...
I20260925 22:05:29.229607 9205 ts_manager.cc:194] Registered new tserver with Master: 6bb97950538540339e7eccb20d86de3a (127.5.0.66:39001)
I20260925 22:05:29.230027 9205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.66@KRBTEST.COM'} at 127.5.0.66:58467
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
I20260925 22:05:29.246517 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.67:0
--local_ip_for_outbound_sockets=127.5.0.67
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:39749
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:42577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:29.327385 9533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:29.327581 9533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:29.327610 9533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:29.328774 9533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:29.328816 9533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:29.328842 9533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.67
I20260925 22:05:29.330291 9533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:42577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:39749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.67
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:29.330511 9533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:29.330775 9533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:29.335444 9538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:29.335565 9533 server_base.cc:1034] running on GCE node
W20260925 22:05:29.335454 9541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:29.335454 9539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:29.336031 9533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:29.336278 9533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:29.337427 9533 hybrid_clock.cc:648] HybridClock initialized: now 1790373929337415 us; error 38 us; skew 500 ppm
Sep 25 22:05:29 dist-test-slave-7mqr krb5kdc[9161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373929, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:29.340131 9533 init.cc:377] Logged in from keytab as kudu/127.5.0.67@KRBTEST.COM (short username kudu)
I20260925 22:05:29.340500 9533 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:29.340566 9533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:29.340695 9533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:29.341701 9533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-2/data/instance:
uuid: "12f7f51006024e59860126029c0ffa79"
format_stamp: "Formatted at 2026-09-25 22:05:29 on dist-test-slave-7mqr"
server_key: "d950f071d6d093a0585ae399590ec957"
server_key_iv: "c263509cd8b20eec480570d4cac0e45f"
server_key_version: "encryptionkey@0"
I20260925 22:05:29.342048 9533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance:
uuid: "12f7f51006024e59860126029c0ffa79"
format_stamp: "Formatted at 2026-09-25 22:05:29 on dist-test-slave-7mqr"
server_key: "d950f071d6d093a0585ae399590ec957"
server_key_iv: "c263509cd8b20eec480570d4cac0e45f"
server_key_version: "encryptionkey@0"
I20260925 22:05:29.343238 9533 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:05:29.343940 9547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:29.344118 9533 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:05:29.344218 9533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-2/wal
uuid: "12f7f51006024e59860126029c0ffa79"
format_stamp: "Formatted at 2026-09-25 22:05:29 on dist-test-slave-7mqr"
server_key: "d950f071d6d093a0585ae399590ec957"
server_key_iv: "c263509cd8b20eec480570d4cac0e45f"
server_key_version: "encryptionkey@0"
I20260925 22:05:29.344296 9533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:29.364634 9533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:29.366930 9533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:29.367085 9533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:29.367329 9533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:29.367641 9533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:29.367690 9533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:29.367733 9533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:29.367758 9533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:29.373566 9533 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.67:45275
I20260925 22:05:29.373601 9660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.67:45275 every 8 connection(s)
I20260925 22:05:29.373878 9533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
Sep 25 22:05:29 dist-test-slave-7mqr krb5kdc[9161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373929, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:29.381407 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 9533
I20260925 22:05:29.381549 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance
I20260925 22:05:29.381740 5121 external_mini_cluster.cc:1428] Setting key f37ada5bfcfab98a7270c9b37324e37d
I20260925 22:05:29.382782 9244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:29.374638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:35475 (local address 127.5.0.126:39749)
0925 22:05:29.374683 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:29.374685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:29.374858 (+ 173us) server_negotiation.cc:407] Connection header received
0925 22:05:29.375312 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:29.375313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:29.375323 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:29.375374 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:29.376304 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.376795 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.377770 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.377944 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.377993 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:29.380108 (+ 2115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:29.380109 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:29.380109 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:29.380116 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:29.381364 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:29.381846 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:29.381847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:29.381847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:29.381911 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:29.382145 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:29.382145 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:29.382146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:29.382286 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:29.382355 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:29.382618 (+ 263us) server_negotiation.cc:299] Negotiation successful
0925 22:05:29.382714 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 22:05:29.382957 9663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:29.374607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:39749 (local address 127.5.0.67:35475)
0925 22:05:29.374757 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:29.374792 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:29.375243 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:29.375428 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:29.375431 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:29.375697 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:29.376229 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:29.376234 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.376896 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.376897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:29.377696 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:29.377697 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.378011 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.378011 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:29.378115 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:29.378652 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:29.378666 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:29.380022 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:29.381439 (+ 1417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:29.381444 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:29.381446 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:29.381743 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:29.381975 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:29.381976 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:29.381976 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:29.382097 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:29.382353 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:29.382358 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:29.382485 (+ 127us) client_negotiation.cc:769] Sending connection context
0925 22:05:29.382598 (+ 113us) client_negotiation.cc:240] Negotiation successful
0925 22:05:29.382754 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":92,"threads_started":1}
I20260925 22:05:29.383347 9661 heartbeater.cc:347] Connected to a master server at 127.5.0.126:39749
I20260925 22:05:29.383433 9661 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:29.383546 9661 heartbeater.cc:511] Master 127.5.0.126:39749 requested a full tablet report, sending...
I20260925 22:05:29.383852 9205 ts_manager.cc:194] Registered new tserver with Master: 12f7f51006024e59860126029c0ffa79 (127.5.0.67:45275)
I20260925 22:05:29.384233 9205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.67@KRBTEST.COM'} at 127.5.0.67:35475
I20260925 22:05:29.393342 5121 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:05:29.400599 9244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:29.394583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51560 (local address 127.5.0.126:39749)
0925 22:05:29.394656 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:29.394659 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:29.394772 (+ 113us) server_negotiation.cc:407] Connection header received
0925 22:05:29.394846 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:29.394847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:29.394857 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:29.394922 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:29.395476 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.396044 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.396801 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.396991 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.397067 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:29.398188 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:29.398189 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:29.398190 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:29.398196 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:29.399434 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:29.399790 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:29.399791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:29.399791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:29.399847 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:29.400051 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:29.400051 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:29.400052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:29.400180 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:29.400319 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:29.400430 (+ 111us) server_negotiation.cc:299] Negotiation successful
0925 22:05:29.400526 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 22:05:29.402225 9205 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51560:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:05:29.403683 9205 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:05:29.409766 9674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:29.406036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46358 (local address 127.5.0.66:39001)
0925 22:05:29.406173 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:29.406179 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:29.406189 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:05:29.406237 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:29.406239 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:29.406254 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:29.406336 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:29.406585 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.407626 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.408833 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.409350 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.409446 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:29.409507 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:29.409645 (+ 138us) server_negotiation.cc:299] Negotiation successful
0925 22:05:29.409685 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":69,"threads_started":1}
I20260925 22:05:29.409761 9670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:29.405676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:45031 (local address 127.0.0.1:60578)
0925 22:05:29.405970 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:29.405981 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:29.406034 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:29.406344 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:29.406345 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:29.406346 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:29.406379 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:29.406380 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.407341 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.407342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:29.408579 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:29.408582 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.409481 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.409482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:29.409544 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:29.409548 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:05:29.409588 (+ 40us) client_negotiation.cc:240] Negotiation successful
0925 22:05:29.409640 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":54,"threads_started":1}
I20260925 22:05:29.409768 9672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:29.405871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:39001 (local address 127.0.0.1:46358)
0925 22:05:29.406026 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:29.406034 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:29.406074 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:29.406456 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:29.406457 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:29.406459 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:29.406508 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:29.406509 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.407744 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.407745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:29.408754 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:29.408756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.409448 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.409449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:29.409530 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:29.409543 (+ 13us) client_negotiation.cc:769] Sending connection context
0925 22:05:29.409629 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 22:05:29.409672 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":37,"thread_start_us":32,"threads_started":1}
I20260925 22:05:29.409991 9669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:29.405642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:45275 (local address 127.0.0.1:47632)
0925 22:05:29.405772 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:29.405811 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:29.405875 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:29.406256 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:29.406258 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:29.406260 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:29.406327 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:29.406329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.407744 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.407745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:29.408861 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:29.408863 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.409777 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.409778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:29.409836 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:29.409840 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:05:29.409892 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 22:05:29.409933 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":49,"thread_start_us":84,"threads_started":1}
I20260925 22:05:29.410163 9671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:29.405806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60578 (local address 127.5.0.65:45031)
0925 22:05:29.406185 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:29.406189 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:29.406198 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:05:29.406237 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:29.406239 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:29.406254 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:29.406337 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:29.406439 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.407241 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.408662 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.409381 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.409957 (+ 576us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:29.410021 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:29.410048 (+ 27us) server_negotiation.cc:299] Negotiation successful
0925 22:05:29.410091 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":77,"threads_started":1}
I20260925 22:05:29.410221 9673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:29.405829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47632 (local address 127.5.0.67:45275)
0925 22:05:29.406025 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:29.406030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:29.406039 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:05:29.406081 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:29.406083 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:29.406098 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:29.406171 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:29.406508 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.407626 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.408942 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.409631 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.409691 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:29.409746 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:29.410087 (+ 341us) server_negotiation.cc:299] Negotiation successful
0925 22:05:29.410128 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":97,"threads_started":1}
I20260925 22:05:29.410934 9595 tablet_service.cc:1467] Processing CreateTablet for tablet 764da048ea9e43c7bdb2bfac702edf1e (DEFAULT_TABLE table=test-table [id=3103bf7e8f954e74bd3eb80f4498bd30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:29.410934 9329 tablet_service.cc:1467] Processing CreateTablet for tablet 764da048ea9e43c7bdb2bfac702edf1e (DEFAULT_TABLE table=test-table [id=3103bf7e8f954e74bd3eb80f4498bd30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:29.411041 9462 tablet_service.cc:1467] Processing CreateTablet for tablet 764da048ea9e43c7bdb2bfac702edf1e (DEFAULT_TABLE table=test-table [id=3103bf7e8f954e74bd3eb80f4498bd30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:29.411280 9595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 764da048ea9e43c7bdb2bfac702edf1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:29.411280 9329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 764da048ea9e43c7bdb2bfac702edf1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:29.411355 9462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 764da048ea9e43c7bdb2bfac702edf1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:29.413900 9675 tablet_bootstrap.cc:492] T 764da048ea9e43c7bdb2bfac702edf1e P 6bb97950538540339e7eccb20d86de3a: Bootstrap starting.
I20260925 22:05:29.414561 9675 tablet_bootstrap.cc:654] T 764da048ea9e43c7bdb2bfac702edf1e P 6bb97950538540339e7eccb20d86de3a: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:29.414716 9677 tablet_bootstrap.cc:492] T 764da048ea9e43c7bdb2bfac702edf1e P 12f7f51006024e59860126029c0ffa79: Bootstrap starting.
I20260925 22:05:29.415213 9675 log.cc:824] T 764da048ea9e43c7bdb2bfac702edf1e P 6bb97950538540339e7eccb20d86de3a: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:29.415395 9677 tablet_bootstrap.cc:654] T 764da048ea9e43c7bdb2bfac702edf1e P 12f7f51006024e59860126029c0ffa79: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:29.416011 9676 tablet_bootstrap.cc:492] T 764da048ea9e43c7bdb2bfac702edf1e P aaa24921abf74771b5f13044b8c6d68b: Bootstrap starting.
I20260925 22:05:29.416006 9677 log.cc:824] T 764da048ea9e43c7bdb2bfac702edf1e P 12f7f51006024e59860126029c0ffa79: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:29.416064 9675 tablet_bootstrap.cc:492] T 764da048ea9e43c7bdb2bfac702edf1e P 6bb97950538540339e7eccb20d86de3a: No bootstrap required, opened a new log
I20260925 22:05:29.416115 9675 ts_tablet_manager.cc:1397] T 764da048ea9e43c7bdb2bfac702edf1e P 6bb97950538540339e7eccb20d86de3a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:05:29.416538 9676 tablet_bootstrap.cc:654] T 764da048ea9e43c7bdb2bfac702edf1e P aaa24921abf74771b5f13044b8c6d68b: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:29.417004 9676 log.cc:824] T 764da048ea9e43c7bdb2bfac702edf1e P aaa24921abf74771b5f13044b8c6d68b: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:29.417671 9676 tablet_bootstrap.cc:492] T 764da048ea9e43c7bdb2bfac702edf1e P aaa24921abf74771b5f13044b8c6d68b: No bootstrap required, opened a new log
I20260925 22:05:29.417760 9676 ts_tablet_manager.cc:1397] T 764da048ea9e43c7bdb2bfac702edf1e P aaa24921abf74771b5f13044b8c6d68b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260925 22:05:29.417778 9675 raft_consensus.cc:348] T 764da048ea9e43c7bdb2bfac702edf1e P 6bb97950538540339e7eccb20d86de3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12f7f51006024e59860126029c0ffa79" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 45275 } } peers { permanent_uuid: "6bb97950538540339e7eccb20d86de3a" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 39001 } } peers { permanent_uuid: "aaa24921abf74771b5f13044b8c6d68b" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45031 } }
I20260925 22:05:29.417944 9675 raft_consensus.cc:374] T 764da048ea9e43c7bdb2bfac702edf1e P 6bb97950538540339e7eccb20d86de3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:29.417982 9675 raft_consensus.cc:729] T 764da048ea9e43c7bdb2bfac702edf1e P 6bb97950538540339e7eccb20d86de3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bb97950538540339e7eccb20d86de3a, State: Initialized, Role: FOLLOWER
I20260925 22:05:29.418097 9675 consensus_queue.cc:260] T 764da048ea9e43c7bdb2bfac702edf1e P 6bb97950538540339e7eccb20d86de3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12f7f51006024e59860126029c0ffa79" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 45275 } } peers { permanent_uuid: "6bb97950538540339e7eccb20d86de3a" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 39001 } } peers { permanent_uuid: "aaa24921abf74771b5f13044b8c6d68b" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45031 } }
I20260925 22:05:29.418366 9675 ts_tablet_manager.cc:1428] T 764da048ea9e43c7bdb2bfac702edf1e P 6bb97950538540339e7eccb20d86de3a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:05:29.418438 9528 heartbeater.cc:503] Master 127.5.0.126:39749 was elected leader, sending a full tablet report...
I20260925 22:05:29.418968 9677 tablet_bootstrap.cc:492] T 764da048ea9e43c7bdb2bfac702edf1e P 12f7f51006024e59860126029c0ffa79: No bootstrap required, opened a new log
I20260925 22:05:29.419039 9677 ts_tablet_manager.cc:1397] T 764da048ea9e43c7bdb2bfac702edf1e P 12f7f51006024e59860126029c0ffa79: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260925 22:05:29.419700 9676 raft_consensus.cc:348] T 764da048ea9e43c7bdb2bfac702edf1e P aaa24921abf74771b5f13044b8c6d68b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12f7f51006024e59860126029c0ffa79" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 45275 } } peers { permanent_uuid: "6bb97950538540339e7eccb20d86de3a" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 39001 } } peers { permanent_uuid: "aaa24921abf74771b5f13044b8c6d68b" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45031 } }
I20260925 22:05:29.419806 9676 raft_consensus.cc:374] T 764da048ea9e43c7bdb2bfac702edf1e P aaa24921abf74771b5f13044b8c6d68b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:29.419826 9676 raft_consensus.cc:729] T 764da048ea9e43c7bdb2bfac702edf1e P aaa24921abf74771b5f13044b8c6d68b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaa24921abf74771b5f13044b8c6d68b, State: Initialized, Role: FOLLOWER
I20260925 22:05:29.419945 9676 consensus_queue.cc:260] T 764da048ea9e43c7bdb2bfac702edf1e P aaa24921abf74771b5f13044b8c6d68b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12f7f51006024e59860126029c0ffa79" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 45275 } } peers { permanent_uuid: "6bb97950538540339e7eccb20d86de3a" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 39001 } } peers { permanent_uuid: "aaa24921abf74771b5f13044b8c6d68b" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45031 } }
I20260925 22:05:29.420217 9676 ts_tablet_manager.cc:1428] T 764da048ea9e43c7bdb2bfac702edf1e P aaa24921abf74771b5f13044b8c6d68b: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260925 22:05:29.420226 9395 heartbeater.cc:503] Master 127.5.0.126:39749 was elected leader, sending a full tablet report...
I20260925 22:05:29.420955 9677 raft_consensus.cc:348] T 764da048ea9e43c7bdb2bfac702edf1e P 12f7f51006024e59860126029c0ffa79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12f7f51006024e59860126029c0ffa79" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 45275 } } peers { permanent_uuid: "6bb97950538540339e7eccb20d86de3a" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 39001 } } peers { permanent_uuid: "aaa24921abf74771b5f13044b8c6d68b" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45031 } }
I20260925 22:05:29.421105 9677 raft_consensus.cc:374] T 764da048ea9e43c7bdb2bfac702edf1e P 12f7f51006024e59860126029c0ffa79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:29.421165 9677 raft_consensus.cc:729] T 764da048ea9e43c7bdb2bfac702edf1e P 12f7f51006024e59860126029c0ffa79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12f7f51006024e59860126029c0ffa79, State: Initialized, Role: FOLLOWER
I20260925 22:05:29.421279 9677 consensus_queue.cc:260] T 764da048ea9e43c7bdb2bfac702edf1e P 12f7f51006024e59860126029c0ffa79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12f7f51006024e59860126029c0ffa79" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 45275 } } peers { permanent_uuid: "6bb97950538540339e7eccb20d86de3a" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 39001 } } peers { permanent_uuid: "aaa24921abf74771b5f13044b8c6d68b" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45031 } }
I20260925 22:05:29.421564 9677 ts_tablet_manager.cc:1428] T 764da048ea9e43c7bdb2bfac702edf1e P 12f7f51006024e59860126029c0ffa79: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:05:29.421564 9661 heartbeater.cc:503] Master 127.5.0.126:39749 was elected leader, sending a full tablet report...
W20260925 22:05:29.468647 9529 tablet.cc:2369] T 764da048ea9e43c7bdb2bfac702edf1e P 6bb97950538540339e7eccb20d86de3a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 22:05:29.566885 9396 tablet.cc:2369] T 764da048ea9e43c7bdb2bfac702edf1e P aaa24921abf74771b5f13044b8c6d68b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 22:05:29.624431 9662 tablet.cc:2369] T 764da048ea9e43c7bdb2bfac702edf1e P 12f7f51006024e59860126029c0ffa79: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:05:29.706985 9681 raft_consensus.cc:482] T 764da048ea9e43c7bdb2bfac702edf1e P 6bb97950538540339e7eccb20d86de3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:05:29.707125 9681 raft_consensus.cc:504] T 764da048ea9e43c7bdb2bfac702edf1e P 6bb97950538540339e7eccb20d86de3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12f7f51006024e59860126029c0ffa79" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 45275 } } peers { permanent_uuid: "6bb97950538540339e7eccb20d86de3a" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 39001 } } peers { permanent_uuid: "aaa24921abf74771b5f13044b8c6d68b" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45031 } }
I20260925 22:05:29.707468 9681 leader_election.cc:288] T 764da048ea9e43c7bdb2bfac702edf1e P 6bb97950538540339e7eccb20d86de3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12f7f51006024e59860126029c0ffa79 (127.5.0.67:45275), aaa24921abf74771b5f13044b8c6d68b (127.5.0.65:45031)
I20260925 22:05:29.710431 9671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:29.707752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:33185 (local address 127.5.0.65:45031)
0925 22:05:29.707820 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:29.707822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:29.707969 (+ 147us) server_negotiation.cc:407] Connection header received
0925 22:05:29.708080 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:29.708081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:29.708093 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:29.708124 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:29.708344 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.708816 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.709702 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.710068 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.710152 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:29.710161 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:29.710329 (+ 168us) server_negotiation.cc:299] Negotiation successful
0925 22:05:29.710370 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 22:05:29.710412 9530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:29.707675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:45275 (local address 127.5.0.66:59751)
0925 22:05:29.707757 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:29.707769 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:29.707817 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:29.707986 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:29.707987 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:29.707989 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:29.708073 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:29.708075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.708796 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.708798 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:29.709600 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:29.709603 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.710127 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.710128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:29.710219 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:29.710222 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:05:29.710270 (+ 48us) client_negotiation.cc:240] Negotiation successful
0925 22:05:29.710321 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260925 22:05:29.710410 9673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:29.707778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:59751 (local address 127.5.0.67:45275)
0925 22:05:29.707838 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:29.707841 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:29.707853 (+ 12us) server_negotiation.cc:407] Connection header received
0925 22:05:29.707882 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:29.707883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:29.707913 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:29.707984 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:29.708149 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.708719 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.709692 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.710050 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.710104 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:29.710113 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:29.710296 (+ 183us) server_negotiation.cc:299] Negotiation successful
0925 22:05:29.710337 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 22:05:29.710431 9684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:29.707675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:45031 (local address 127.5.0.66:33185)
0925 22:05:29.707846 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:29.707857 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:29.707997 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:29.708195 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:29.708195 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:29.708197 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:29.708270 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:29.708272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.708894 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.708895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:29.709605 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:29.709606 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.710163 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.710164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:29.710237 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:29.710240 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:05:29.710313 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 22:05:29.710346 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":12,"thread_start_us":74,"threads_started":1}
I20260925 22:05:29.710798 9615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "764da048ea9e43c7bdb2bfac702edf1e" candidate_uuid: "6bb97950538540339e7eccb20d86de3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12f7f51006024e59860126029c0ffa79" is_pre_election: true
I20260925 22:05:29.710959 9615 raft_consensus.cc:2454] T 764da048ea9e43c7bdb2bfac702edf1e P 12f7f51006024e59860126029c0ffa79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bb97950538540339e7eccb20d86de3a in term 0.
I20260925 22:05:29.710978 9349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "764da048ea9e43c7bdb2bfac702edf1e" candidate_uuid: "6bb97950538540339e7eccb20d86de3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aaa24921abf74771b5f13044b8c6d68b" is_pre_election: true
I20260925 22:05:29.711135 9349 raft_consensus.cc:2454] T 764da048ea9e43c7bdb2bfac702edf1e P aaa24921abf74771b5f13044b8c6d68b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bb97950538540339e7eccb20d86de3a in term 0.
I20260925 22:05:29.711197 9418 leader_election.cc:302] T 764da048ea9e43c7bdb2bfac702edf1e P 6bb97950538540339e7eccb20d86de3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12f7f51006024e59860126029c0ffa79, 6bb97950538540339e7eccb20d86de3a; no voters:
I20260925 22:05:29.711304 9681 raft_consensus.cc:2781] T 764da048ea9e43c7bdb2bfac702edf1e P 6bb97950538540339e7eccb20d86de3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:05:29.711355 9681 raft_consensus.cc:482] T 764da048ea9e43c7bdb2bfac702edf1e P 6bb97950538540339e7eccb20d86de3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:05:29.711385 9681 raft_consensus.cc:3037] T 764da048ea9e43c7bdb2bfac702edf1e P 6bb97950538540339e7eccb20d86de3a [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:29.712076 9681 raft_consensus.cc:504] T 764da048ea9e43c7bdb2bfac702edf1e P 6bb97950538540339e7eccb20d86de3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12f7f51006024e59860126029c0ffa79" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 45275 } } peers { permanent_uuid: "6bb97950538540339e7eccb20d86de3a" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 39001 } } peers { permanent_uuid: "aaa24921abf74771b5f13044b8c6d68b" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45031 } }
I20260925 22:05:29.712224 9681 leader_election.cc:288] T 764da048ea9e43c7bdb2bfac702edf1e P 6bb97950538540339e7eccb20d86de3a [CANDIDATE]: Term 1 election: Requested vote from peers 12f7f51006024e59860126029c0ffa79 (127.5.0.67:45275), aaa24921abf74771b5f13044b8c6d68b (127.5.0.65:45031)
I20260925 22:05:29.712363 9349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "764da048ea9e43c7bdb2bfac702edf1e" candidate_uuid: "6bb97950538540339e7eccb20d86de3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aaa24921abf74771b5f13044b8c6d68b"
I20260925 22:05:29.712380 9615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "764da048ea9e43c7bdb2bfac702edf1e" candidate_uuid: "6bb97950538540339e7eccb20d86de3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12f7f51006024e59860126029c0ffa79"
I20260925 22:05:29.712435 9349 raft_consensus.cc:3037] T 764da048ea9e43c7bdb2bfac702edf1e P aaa24921abf74771b5f13044b8c6d68b [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:29.712462 9615 raft_consensus.cc:3037] T 764da048ea9e43c7bdb2bfac702edf1e P 12f7f51006024e59860126029c0ffa79 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:29.713131 9349 raft_consensus.cc:2454] T 764da048ea9e43c7bdb2bfac702edf1e P aaa24921abf74771b5f13044b8c6d68b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bb97950538540339e7eccb20d86de3a in term 1.
I20260925 22:05:29.713151 9615 raft_consensus.cc:2454] T 764da048ea9e43c7bdb2bfac702edf1e P 12f7f51006024e59860126029c0ffa79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bb97950538540339e7eccb20d86de3a in term 1.
I20260925 22:05:29.713310 9418 leader_election.cc:302] T 764da048ea9e43c7bdb2bfac702edf1e P 6bb97950538540339e7eccb20d86de3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12f7f51006024e59860126029c0ffa79, 6bb97950538540339e7eccb20d86de3a; no voters:
I20260925 22:05:29.713444 9681 raft_consensus.cc:2781] T 764da048ea9e43c7bdb2bfac702edf1e P 6bb97950538540339e7eccb20d86de3a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:29.713609 9681 raft_consensus.cc:686] T 764da048ea9e43c7bdb2bfac702edf1e P 6bb97950538540339e7eccb20d86de3a [term 1 LEADER]: Becoming Leader. State: Replica: 6bb97950538540339e7eccb20d86de3a, State: Running, Role: LEADER
I20260925 22:05:29.713717 9681 consensus_queue.cc:237] T 764da048ea9e43c7bdb2bfac702edf1e P 6bb97950538540339e7eccb20d86de3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12f7f51006024e59860126029c0ffa79" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 45275 } } peers { permanent_uuid: "6bb97950538540339e7eccb20d86de3a" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 39001 } } peers { permanent_uuid: "aaa24921abf74771b5f13044b8c6d68b" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45031 } }
I20260925 22:05:29.714454 9205 catalog_manager.cc:5495] T 764da048ea9e43c7bdb2bfac702edf1e P 6bb97950538540339e7eccb20d86de3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 6bb97950538540339e7eccb20d86de3a (127.5.0.66). New cstate: current_term: 1 leader_uuid: "6bb97950538540339e7eccb20d86de3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12f7f51006024e59860126029c0ffa79" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bb97950538540339e7eccb20d86de3a" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 39001 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aaa24921abf74771b5f13044b8c6d68b" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45031 } health_report { overall_health: UNKNOWN } } }
I20260925 22:05:29.762343 9674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:29.760227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46372 (local address 127.5.0.66:39001)
0925 22:05:29.760290 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:29.760292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:29.760301 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:05:29.760339 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:29.760340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:29.760362 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:29.760423 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:05:29.760555 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.761014 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.761708 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:29.761881 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:29.761930 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:29.762059 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:05:29.762154 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:05:29.762220 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:29.762250 (+ 30us) server_negotiation.cc:299] Negotiation successful
0925 22:05:29.762267 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 22:05:29.764081 9615 raft_consensus.cc:1261] T 764da048ea9e43c7bdb2bfac702edf1e P 12f7f51006024e59860126029c0ffa79 [term 1 FOLLOWER]: Refusing update from remote peer 6bb97950538540339e7eccb20d86de3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:05:29.764137 9349 raft_consensus.cc:1261] T 764da048ea9e43c7bdb2bfac702edf1e P aaa24921abf74771b5f13044b8c6d68b [term 1 FOLLOWER]: Refusing update from remote peer 6bb97950538540339e7eccb20d86de3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:05:29.764463 9685 consensus_queue.cc:1035] T 764da048ea9e43c7bdb2bfac702edf1e P 6bb97950538540339e7eccb20d86de3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "aaa24921abf74771b5f13044b8c6d68b" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:05:29.764591 9681 consensus_queue.cc:1035] T 764da048ea9e43c7bdb2bfac702edf1e P 6bb97950538540339e7eccb20d86de3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12f7f51006024e59860126029c0ffa79" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:05:29.767817 9689 mvcc.cc:205] Tried to move back new op lower bound from 7333371616311074816 to 7333371616107954176. Current Snapshot: MvccSnapshot[applied={T|T < 7333371616311074816}]
I20260925 22:05:29.769142 9204 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:05:29.769227 9204 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:05:29.769945 9204 catalog_manager.cc:5782] T 00000000000000000000000000000000 P df7f04264e474c6d9f99d2a181e709a8: Sending DeleteTablet for 3 replicas of tablet 764da048ea9e43c7bdb2bfac702edf1e
I20260925 22:05:29.770244 9595 tablet_service.cc:1514] Processing DeleteTablet for tablet 764da048ea9e43c7bdb2bfac702edf1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:29 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:47632
I20260925 22:05:29.770298 9462 tablet_service.cc:1514] Processing DeleteTablet for tablet 764da048ea9e43c7bdb2bfac702edf1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:29 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:46358
I20260925 22:05:29.770464 9329 tablet_service.cc:1514] Processing DeleteTablet for tablet 764da048ea9e43c7bdb2bfac702edf1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:29 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:60578
I20260925 22:05:29.770507 9699 tablet_replica.cc:330] T 764da048ea9e43c7bdb2bfac702edf1e P 6bb97950538540339e7eccb20d86de3a: stopping tablet replica
I20260925 22:05:29.770601 9699 raft_consensus.cc:2229] T 764da048ea9e43c7bdb2bfac702edf1e P 6bb97950538540339e7eccb20d86de3a [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:05:29.770689 9699 raft_consensus.cc:2258] T 764da048ea9e43c7bdb2bfac702edf1e P 6bb97950538540339e7eccb20d86de3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:29.770718 9700 tablet_replica.cc:330] T 764da048ea9e43c7bdb2bfac702edf1e P aaa24921abf74771b5f13044b8c6d68b: stopping tablet replica
I20260925 22:05:29.770813 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 9267
I20260925 22:05:29.770838 9700 raft_consensus.cc:2229] T 764da048ea9e43c7bdb2bfac702edf1e P aaa24921abf74771b5f13044b8c6d68b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:05:29.770942 9700 raft_consensus.cc:2258] T 764da048ea9e43c7bdb2bfac702edf1e P aaa24921abf74771b5f13044b8c6d68b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:29.771368 9698 tablet_replica.cc:330] T 764da048ea9e43c7bdb2bfac702edf1e P 12f7f51006024e59860126029c0ffa79: stopping tablet replica
I20260925 22:05:29.771461 9698 raft_consensus.cc:2229] T 764da048ea9e43c7bdb2bfac702edf1e P 12f7f51006024e59860126029c0ffa79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:05:29.771565 9698 raft_consensus.cc:2258] T 764da048ea9e43c7bdb2bfac702edf1e P 12f7f51006024e59860126029c0ffa79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:29.771878 9699 ts_tablet_manager.cc:1905] T 764da048ea9e43c7bdb2bfac702edf1e P 6bb97950538540339e7eccb20d86de3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:29.771993 9698 ts_tablet_manager.cc:1905] T 764da048ea9e43c7bdb2bfac702edf1e P 12f7f51006024e59860126029c0ffa79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:29.773586 9699 ts_tablet_manager.cc:1918] T 764da048ea9e43c7bdb2bfac702edf1e P 6bb97950538540339e7eccb20d86de3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:05:29.773684 9699 log.cc:1196] T 764da048ea9e43c7bdb2bfac702edf1e P 6bb97950538540339e7eccb20d86de3a: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-1/wal/wals/764da048ea9e43c7bdb2bfac702edf1e
I20260925 22:05:29.773880 9698 ts_tablet_manager.cc:1918] T 764da048ea9e43c7bdb2bfac702edf1e P 12f7f51006024e59860126029c0ffa79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:05:29.773974 9698 log.cc:1196] T 764da048ea9e43c7bdb2bfac702edf1e P 12f7f51006024e59860126029c0ffa79: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373908987421-5121-0/minicluster-data/ts-2/wal/wals/764da048ea9e43c7bdb2bfac702edf1e
I20260925 22:05:29.773985 9699 ts_tablet_manager.cc:1939] T 764da048ea9e43c7bdb2bfac702edf1e P 6bb97950538540339e7eccb20d86de3a: Deleting consensus metadata
I20260925 22:05:29.774271 9698 ts_tablet_manager.cc:1939] T 764da048ea9e43c7bdb2bfac702edf1e P 12f7f51006024e59860126029c0ffa79: Deleting consensus metadata
I20260925 22:05:29.775027 9192 catalog_manager.cc:4841] TS 12f7f51006024e59860126029c0ffa79 (127.5.0.67:45275): tablet 764da048ea9e43c7bdb2bfac702edf1e (table test-table [id=3103bf7e8f954e74bd3eb80f4498bd30]) successfully deleted
I20260925 22:05:29.775132 9192 catalog_manager.cc:4841] TS 6bb97950538540339e7eccb20d86de3a (127.5.0.66:39001): tablet 764da048ea9e43c7bdb2bfac702edf1e (table test-table [id=3103bf7e8f954e74bd3eb80f4498bd30]) successfully deleted
W20260925 22:05:29.778183 9189 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.0.65:45031 (error 108)
I20260925 22:05:29.778306 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 9400
I20260925 22:05:29.778664 9669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:29.778548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:45031 (local address 127.0.0.1:60594)
0925 22:05:29.778595 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:29.778624 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.65:45031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260925 22:05:29.778770 9189 catalog_manager.cc:4584] TS aaa24921abf74771b5f13044b8c6d68b (127.5.0.65:45031): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 764da048ea9e43c7bdb2bfac702edf1e: Network error: Client connection negotiation failed: client connection to 127.5.0.65:45031: connect: Connection refused (error 111)
I20260925 22:05:29.782713 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 9533
I20260925 22:05:29.787515 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 9174
2026-09-25T22:05:29Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2160 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-25T22:05:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:05:29Z Disabled control of system clock
I20260925 22:05:29.814658 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:46765
--webserver_interface=127.5.0.126
--webserver_port=0
--builtin_ntp_servers=127.5.0.84:33813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:46765
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 22:05:29.897253 9705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:29.897514 9705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:29.897555 9705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:29.898823 9705 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:05:29.899073 9705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:29.899147 9705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:29.899173 9705 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:05:29.899199 9705 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:05:29.900800 9705 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:33813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:46765
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:46765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:29.901080 9705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:29.901362 9705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:29.906102 9711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:29.906095 9710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:29.906096 9713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:29.906539 9705 server_base.cc:1034] running on GCE node
I20260925 22:05:29.906791 9705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:29.907029 9705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:29.908159 9705 hybrid_clock.cc:648] HybridClock initialized: now 1790373929908148 us; error 27 us; skew 500 ppm
I20260925 22:05:29.909384 9705 webserver.cc:460] Webserver started at http://127.5.0.126:45997/ using document root <none> and password file <none>
I20260925 22:05:29.909619 9705 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:29.909667 9705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:29.909834 9705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:29.910880 9705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "44782d782dbd4db89c732b3d12bfa5ce"
format_stamp: "Formatted at 2026-09-25 22:05:29 on dist-test-slave-7mqr"
server_key: "17100c72dda013c104811a038974bb20"
server_key_iv: "8dbb03063156643267ba43d206d426fd"
server_key_version: "encryptionkey@0"
I20260925 22:05:29.911242 9705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "44782d782dbd4db89c732b3d12bfa5ce"
format_stamp: "Formatted at 2026-09-25 22:05:29 on dist-test-slave-7mqr"
server_key: "17100c72dda013c104811a038974bb20"
server_key_iv: "8dbb03063156643267ba43d206d426fd"
server_key_version: "encryptionkey@0"
I20260925 22:05:29.912587 9705 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:05:29.913281 9719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:29.913444 9705 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:05:29.913542 9705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "44782d782dbd4db89c732b3d12bfa5ce"
format_stamp: "Formatted at 2026-09-25 22:05:29 on dist-test-slave-7mqr"
server_key: "17100c72dda013c104811a038974bb20"
server_key_iv: "8dbb03063156643267ba43d206d426fd"
server_key_version: "encryptionkey@0"
I20260925 22:05:29.913610 9705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:29.916908 9705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:29.917162 9705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:29.917299 9705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:29.920954 9705 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:46765
I20260925 22:05:29.920982 9771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:46765 every 8 connection(s)
I20260925 22:05:29.921289 9705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:05:29.921875 9772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:29.924604 9772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44782d782dbd4db89c732b3d12bfa5ce: Bootstrap starting.
I20260925 22:05:29.925176 9772 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44782d782dbd4db89c732b3d12bfa5ce: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:29.925477 9772 log.cc:824] T 00000000000000000000000000000000 P 44782d782dbd4db89c732b3d12bfa5ce: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:29.926118 9772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44782d782dbd4db89c732b3d12bfa5ce: No bootstrap required, opened a new log
I20260925 22:05:29.927693 9772 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44782d782dbd4db89c732b3d12bfa5ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44782d782dbd4db89c732b3d12bfa5ce" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 46765 } }
I20260925 22:05:29.927834 9772 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44782d782dbd4db89c732b3d12bfa5ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:29.927856 9772 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44782d782dbd4db89c732b3d12bfa5ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44782d782dbd4db89c732b3d12bfa5ce, State: Initialized, Role: FOLLOWER
I20260925 22:05:29.928061 9772 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44782d782dbd4db89c732b3d12bfa5ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44782d782dbd4db89c732b3d12bfa5ce" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 46765 } }
I20260925 22:05:29.928140 9772 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44782d782dbd4db89c732b3d12bfa5ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:05:29.928179 9772 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44782d782dbd4db89c732b3d12bfa5ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:05:29.928237 9772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44782d782dbd4db89c732b3d12bfa5ce [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:29.928535 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 9705
I20260925 22:05:29.928632 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:05:29.928822 5121 external_mini_cluster.cc:1428] Setting key 3d3a2658f78a39eb2eab3029a35e910a
I20260925 22:05:29.929003 9772 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44782d782dbd4db89c732b3d12bfa5ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44782d782dbd4db89c732b3d12bfa5ce" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 46765 } }
I20260925 22:05:29.929129 9772 leader_election.cc:302] T 00000000000000000000000000000000 P 44782d782dbd4db89c732b3d12bfa5ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44782d782dbd4db89c732b3d12bfa5ce; no voters:
I20260925 22:05:29.929309 9772 leader_election.cc:288] T 00000000000000000000000000000000 P 44782d782dbd4db89c732b3d12bfa5ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:05:29.929462 9776 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44782d782dbd4db89c732b3d12bfa5ce [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:29.929575 9776 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44782d782dbd4db89c732b3d12bfa5ce [term 1 LEADER]: Becoming Leader. State: Replica: 44782d782dbd4db89c732b3d12bfa5ce, State: Running, Role: LEADER
I20260925 22:05:29.929726 9776 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44782d782dbd4db89c732b3d12bfa5ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44782d782dbd4db89c732b3d12bfa5ce" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 46765 } }
I20260925 22:05:29.930094 9776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44782d782dbd4db89c732b3d12bfa5ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44782d782dbd4db89c732b3d12bfa5ce. Latest consensus state: current_term: 1 leader_uuid: "44782d782dbd4db89c732b3d12bfa5ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44782d782dbd4db89c732b3d12bfa5ce" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 46765 } } }
I20260925 22:05:29.930164 9776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44782d782dbd4db89c732b3d12bfa5ce [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:29.929576 9772 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44782d782dbd4db89c732b3d12bfa5ce [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:05:29.930333 9779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44782d782dbd4db89c732b3d12bfa5ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44782d782dbd4db89c732b3d12bfa5ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44782d782dbd4db89c732b3d12bfa5ce" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 46765 } } }
I20260925 22:05:29.930500 9779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44782d782dbd4db89c732b3d12bfa5ce [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:29.930609 9781 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:05:29.931198 9781 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:05:29.931252 9777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:29.929278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50042 (local address 127.5.0.126:46765)
0925 22:05:29.929509 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:29.929518 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:29.929553 (+ 35us) server_negotiation.cc:407] Connection header received
0925 22:05:29.929665 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:29.929672 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:29.929732 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:29.929891 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:29.930618 (+ 727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:29.930621 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:29.930622 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:05:29.930625 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:29.930663 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:05:29.930690 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:29.930973 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:29.931006 (+ 33us) server_negotiation.cc:299] Negotiation successful
0925 22:05:29.931066 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":66,"threads_started":1}
I20260925 22:05:29.932833 9781 catalog_manager.cc:1347] Generated new cluster ID: 89e5d152387c4844b1691c850fc2f45c
I20260925 22:05:29.932888 9781 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:05:29.944850 9781 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:05:29.945026 9781 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:05:29.956139 9781 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44782d782dbd4db89c732b3d12bfa5ce: Generated new TSK 0
I20260925 22:05:29.956365 9781 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:05:29.957456 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.65:0
--local_ip_for_outbound_sockets=127.5.0.65
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:46765
--builtin_ntp_servers=127.5.0.84:33813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 22:05:30.039397 9796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:30.039618 9796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:30.039655 9796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:30.041092 9796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:30.041157 9796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:30.041209 9796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.65
I20260925 22:05:30.042541 9796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:33813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:46765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.65
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:30.042798 9796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:30.043085 9796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:30.047984 9802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:30.048038 9804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:30.048259 9801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:30.048322 9796 server_base.cc:1034] running on GCE node
I20260925 22:05:30.048547 9796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:30.048775 9796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:30.049926 9796 hybrid_clock.cc:648] HybridClock initialized: now 1790373930049912 us; error 32 us; skew 500 ppm
I20260925 22:05:30.051141 9796 webserver.cc:460] Webserver started at http://127.5.0.65:37189/ using document root <none> and password file <none>
I20260925 22:05:30.051388 9796 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:30.051455 9796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:30.051589 9796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:30.052690 9796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-0/data/instance:
uuid: "b56837df97c8422eae538ec66db22116"
format_stamp: "Formatted at 2026-09-25 22:05:30 on dist-test-slave-7mqr"
server_key: "a1bd1ca10ac1bbc83074afaa5002e53f"
server_key_iv: "745684ccb44af31b201d00f269053409"
server_key_version: "encryptionkey@0"
I20260925 22:05:30.053037 9796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance:
uuid: "b56837df97c8422eae538ec66db22116"
format_stamp: "Formatted at 2026-09-25 22:05:30 on dist-test-slave-7mqr"
server_key: "a1bd1ca10ac1bbc83074afaa5002e53f"
server_key_iv: "745684ccb44af31b201d00f269053409"
server_key_version: "encryptionkey@0"
I20260925 22:05:30.054317 9796 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 22:05:30.055033 9810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:30.055198 9796 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:05:30.055301 9796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-0/wal
uuid: "b56837df97c8422eae538ec66db22116"
format_stamp: "Formatted at 2026-09-25 22:05:30 on dist-test-slave-7mqr"
server_key: "a1bd1ca10ac1bbc83074afaa5002e53f"
server_key_iv: "745684ccb44af31b201d00f269053409"
server_key_version: "encryptionkey@0"
I20260925 22:05:30.055373 9796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:30.058583 9796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:30.058831 9796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:30.058959 9796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:30.059170 9796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:30.059456 9796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:30.059486 9796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:30.059535 9796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:30.059561 9796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:30.065165 9796 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.65:39733
I20260925 22:05:30.065207 9923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.65:39733 every 8 connection(s)
I20260925 22:05:30.065501 9796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
I20260925 22:05:30.068223 9777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:30.066134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:34629 (local address 127.5.0.126:46765)
0925 22:05:30.066178 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:30.066181 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:30.066448 (+ 267us) server_negotiation.cc:407] Connection header received
0925 22:05:30.066892 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:30.066893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:30.066899 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:30.066951 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:30.068035 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:30.068036 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:30.068036 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:05:30.068040 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:30.068053 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:05:30.068057 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:30.068108 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:30.068156 (+ 48us) server_negotiation.cc:299] Negotiation successful
0925 22:05:30.068168 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 22:05:30.068434 9926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:30.066142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:46765 (local address 127.5.0.65:34629)
0925 22:05:30.066333 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:30.066365 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:30.066813 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:30.067000 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:30.067004 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:30.067011 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:30.067850 (+ 839us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:05:30.067851 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:30.067877 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:05:30.067950 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:30.068109 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:30.068111 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:30.068113 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 22:05:30.068163 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 22:05:30.068210 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":103,"threads_started":1}
I20260925 22:05:30.068902 9924 heartbeater.cc:347] Connected to a master server at 127.5.0.126:46765
I20260925 22:05:30.068975 9924 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:30.069113 9924 heartbeater.cc:511] Master 127.5.0.126:46765 requested a full tablet report, sending...
I20260925 22:05:30.069581 9736 ts_manager.cc:194] Registered new tserver with Master: b56837df97c8422eae538ec66db22116 (127.5.0.65:39733)
I20260925 22:05:30.072587 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 9796
I20260925 22:05:30.072695 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance
I20260925 22:05:30.072896 5121 external_mini_cluster.cc:1428] Setting key 8b97368b20eb91e21a5e85807a28cf15
I20260925 22:05:30.073719 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.66:0
--local_ip_for_outbound_sockets=127.5.0.66
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:46765
--builtin_ntp_servers=127.5.0.84:33813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 22:05:30.152899 9927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:30.153100 9927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:30.153121 9927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:30.154259 9927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:30.154297 9927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:30.154345 9927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.66
I20260925 22:05:30.155673 9927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:33813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:46765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.66
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:30.155957 9927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:30.156245 9927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:30.161108 9932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:30.161159 9933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:30.161111 9935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:30.161422 9927 server_base.cc:1034] running on GCE node
I20260925 22:05:30.161605 9927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:30.161800 9927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:30.162954 9927 hybrid_clock.cc:648] HybridClock initialized: now 1790373930162945 us; error 33 us; skew 500 ppm
I20260925 22:05:30.170643 9927 webserver.cc:460] Webserver started at http://127.5.0.66:33813/ using document root <none> and password file <none>
I20260925 22:05:30.170912 9927 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:30.170979 9927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:30.171118 9927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:30.172238 9927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-1/data/instance:
uuid: "32dcd0b6f3384455a4d1ea3a77781c6c"
format_stamp: "Formatted at 2026-09-25 22:05:30 on dist-test-slave-7mqr"
server_key: "6ec595b10dd8ba9c4567841241ea4ee5"
server_key_iv: "ce3648a79b8307a6ae8ee46befb2080a"
server_key_version: "encryptionkey@0"
I20260925 22:05:30.172590 9927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance:
uuid: "32dcd0b6f3384455a4d1ea3a77781c6c"
format_stamp: "Formatted at 2026-09-25 22:05:30 on dist-test-slave-7mqr"
server_key: "6ec595b10dd8ba9c4567841241ea4ee5"
server_key_iv: "ce3648a79b8307a6ae8ee46befb2080a"
server_key_version: "encryptionkey@0"
I20260925 22:05:30.173838 9927 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:05:30.174508 9941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:30.174664 9927 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:05:30.174770 9927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-1/wal
uuid: "32dcd0b6f3384455a4d1ea3a77781c6c"
format_stamp: "Formatted at 2026-09-25 22:05:30 on dist-test-slave-7mqr"
server_key: "6ec595b10dd8ba9c4567841241ea4ee5"
server_key_iv: "ce3648a79b8307a6ae8ee46befb2080a"
server_key_version: "encryptionkey@0"
I20260925 22:05:30.174839 9927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:30.178188 9927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:30.178442 9927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:30.178580 9927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:30.178781 9927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:30.179066 9927 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:30.179116 9927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:30.179155 9927 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:30.179183 9927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:30.184737 9927 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.66:43521
I20260925 22:05:30.184777 10054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.66:43521 every 8 connection(s)
I20260925 22:05:30.185081 9927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
I20260925 22:05:30.187793 9777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:30.185703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:50069 (local address 127.5.0.126:46765)
0925 22:05:30.185781 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:30.185783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:30.186037 (+ 254us) server_negotiation.cc:407] Connection header received
0925 22:05:30.186483 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:30.186484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:30.186489 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:30.186543 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:30.187577 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:30.187578 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:30.187578 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:05:30.187582 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:30.187596 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:05:30.187600 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:30.187646 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:30.187729 (+ 83us) server_negotiation.cc:299] Negotiation successful
0925 22:05:30.187741 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 22:05:30.188011 10057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:30.185702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:46765 (local address 127.5.0.66:50069)
0925 22:05:30.185906 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:30.185958 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:30.186394 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:30.186599 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:30.186602 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:30.186609 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:30.187450 (+ 841us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:05:30.187451 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:30.187477 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:05:30.187503 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:30.187653 (+ 150us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:30.187657 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:30.187659 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 22:05:30.187723 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 22:05:30.187769 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
I20260925 22:05:30.188093 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 9927
I20260925 22:05:30.188197 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance
I20260925 22:05:30.188393 5121 external_mini_cluster.cc:1428] Setting key 44efbf9b27f290b66f4dae386bc064cf
I20260925 22:05:30.188895 10055 heartbeater.cc:347] Connected to a master server at 127.5.0.126:46765
I20260925 22:05:30.188989 10055 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:30.189133 10055 heartbeater.cc:511] Master 127.5.0.126:46765 requested a full tablet report, sending...
I20260925 22:05:30.189317 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.67:0
--local_ip_for_outbound_sockets=127.5.0.67
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:46765
--builtin_ntp_servers=127.5.0.84:33813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260925 22:05:30.189512 9736 ts_manager.cc:194] Registered new tserver with Master: 32dcd0b6f3384455a4d1ea3a77781c6c (127.5.0.66:43521)
W20260925 22:05:30.270324 10058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:30.270555 10058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:30.270577 10058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:30.271714 10058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:30.271751 10058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:30.271777 10058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.67
I20260925 22:05:30.273265 10058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:33813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:46765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.67
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:30.273507 10058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:30.273765 10058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:30.278597 10066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:30.278607 10063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:30.278779 10058 server_base.cc:1034] running on GCE node
W20260925 22:05:30.278599 10064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:30.279047 10058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:30.279284 10058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:30.280441 10058 hybrid_clock.cc:648] HybridClock initialized: now 1790373930280429 us; error 35 us; skew 500 ppm
I20260925 22:05:30.281564 10058 webserver.cc:460] Webserver started at http://127.5.0.67:43733/ using document root <none> and password file <none>
I20260925 22:05:30.281787 10058 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:30.281852 10058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:30.282003 10058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:30.283002 10058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-2/data/instance:
uuid: "94e5320e6f16477cadc38c2c19443143"
format_stamp: "Formatted at 2026-09-25 22:05:30 on dist-test-slave-7mqr"
server_key: "3b8edcf76457fb77ad5b608a68d7a2e5"
server_key_iv: "7512cdbdc12f5ea9fc85997fac7e65e1"
server_key_version: "encryptionkey@0"
I20260925 22:05:30.283356 10058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance:
uuid: "94e5320e6f16477cadc38c2c19443143"
format_stamp: "Formatted at 2026-09-25 22:05:30 on dist-test-slave-7mqr"
server_key: "3b8edcf76457fb77ad5b608a68d7a2e5"
server_key_iv: "7512cdbdc12f5ea9fc85997fac7e65e1"
server_key_version: "encryptionkey@0"
I20260925 22:05:30.284690 10058 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:05:30.285348 10072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:30.285506 10058 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:05:30.285597 10058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-2/wal
uuid: "94e5320e6f16477cadc38c2c19443143"
format_stamp: "Formatted at 2026-09-25 22:05:30 on dist-test-slave-7mqr"
server_key: "3b8edcf76457fb77ad5b608a68d7a2e5"
server_key_iv: "7512cdbdc12f5ea9fc85997fac7e65e1"
server_key_version: "encryptionkey@0"
I20260925 22:05:30.285668 10058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:30.288914 10058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:30.289144 10058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:30.289283 10058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:30.289477 10058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:30.289750 10058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:30.289798 10058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:30.289835 10058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:30.289862 10058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:30.295260 10058 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.67:43807
I20260925 22:05:30.295326 10185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.67:43807 every 8 connection(s)
I20260925 22:05:30.295619 10058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
I20260925 22:05:30.298391 9777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:30.296304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:55277 (local address 127.5.0.126:46765)
0925 22:05:30.296366 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:30.296368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:30.296587 (+ 219us) server_negotiation.cc:407] Connection header received
0925 22:05:30.297047 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:30.297048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:30.297054 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:30.297113 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:30.298154 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:30.298156 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:30.298156 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:05:30.298161 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:30.298176 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:05:30.298180 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:30.298227 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:30.298307 (+ 80us) server_negotiation.cc:299] Negotiation successful
0925 22:05:30.298324 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 22:05:30.298574 10188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:30.296299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:46765 (local address 127.5.0.67:55277)
0925 22:05:30.296476 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:30.296505 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:30.296945 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:30.297184 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:30.297188 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:30.297197 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:30.298016 (+ 819us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:05:30.298016 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:30.298041 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:05:30.298065 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:30.298234 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:30.298238 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:30.298241 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:05:30.298308 (+ 67us) client_negotiation.cc:240] Negotiation successful
0925 22:05:30.298367 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":80,"threads_started":1}
I20260925 22:05:30.299032 10186 heartbeater.cc:347] Connected to a master server at 127.5.0.126:46765
I20260925 22:05:30.299101 10186 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:30.299259 10186 heartbeater.cc:511] Master 127.5.0.126:46765 requested a full tablet report, sending...
I20260925 22:05:30.299595 9736 ts_manager.cc:194] Registered new tserver with Master: 94e5320e6f16477cadc38c2c19443143 (127.5.0.67:43807)
I20260925 22:05:30.304032 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 10058
I20260925 22:05:30.304126 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance
I20260925 22:05:30.304257 5121 external_mini_cluster.cc:1428] Setting key 11a4f6dd4e7dd15d87714aa042fd88cf
I20260925 22:05:30.305069 5121 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:05:30.307322 9777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:30.306486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50050 (local address 127.5.0.126:46765)
0925 22:05:30.306523 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:30.306524 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:30.306533 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:05:30.306588 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:30.306588 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:30.306592 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:30.306638 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:30.307140 (+ 502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:30.307141 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:30.307142 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:05:30.307146 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:30.307168 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:05:30.307172 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:30.307231 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:30.307266 (+ 35us) server_negotiation.cc:299] Negotiation successful
0925 22:05:30.307277 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 22:05:30.308799 9736 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50050:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:05:30.310281 9736 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:05:30.313831 10196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:30.312344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46498 (local address 127.5.0.65:39733)
0925 22:05:30.312601 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:30.312605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:30.312718 (+ 113us) server_negotiation.cc:407] Connection header received
0925 22:05:30.312749 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:30.312751 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:30.312761 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:30.312824 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:30.313531 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:30.313539 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:30.313539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:05:30.313544 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:30.313580 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:05:30.313600 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:30.313665 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:30.313760 (+ 95us) server_negotiation.cc:299] Negotiation successful
0925 22:05:30.313772 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":98,"threads_started":1}
I20260925 22:05:30.313856 10195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:30.312255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:39733 (local address 127.0.0.1:46498)
0925 22:05:30.312547 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:30.312591 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:30.312653 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:30.312838 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:30.312839 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:30.312842 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:30.313428 (+ 586us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:05:30.313428 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:30.313435 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:05:30.313445 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:30.313668 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:30.313674 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:30.313677 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:05:30.313758 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 22:05:30.313771 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"spinlock_wait_cycles":384,"thread_start_us":90,"threads_started":1}
I20260925 22:05:30.314050 10194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:30.312255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:43807 (local address 127.0.0.1:39426)
0925 22:05:30.312578 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:30.312592 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:30.312647 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:30.312809 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:30.312813 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:30.312819 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:30.313269 (+ 450us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:05:30.313269 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:30.313286 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:05:30.313310 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:30.313936 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:30.313937 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:30.313938 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 22:05:30.313987 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 22:05:30.313997 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":2,"thread_start_us":57,"threads_started":1}
I20260925 22:05:30.314232 10197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:30.312344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39426 (local address 127.5.0.67:43807)
0925 22:05:30.312680 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:30.312683 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:30.312699 (+ 16us) server_negotiation.cc:407] Connection header received
0925 22:05:30.312724 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:30.312726 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:30.312734 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:30.312804 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:30.313552 (+ 748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:30.313554 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:30.313555 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:05:30.313558 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:30.313584 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:05:30.313598 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:30.313665 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:30.314138 (+ 473us) server_negotiation.cc:299] Negotiation successful
0925 22:05:30.314154 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":95,"threads_started":1}
I20260925 22:05:30.314584 9858 tablet_service.cc:1467] Processing CreateTablet for tablet 475784a803d84ffca4ee5c685117841d (DEFAULT_TABLE table=test-table [id=23223d09e9d840b4a62170c302ee5a61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:30.314755 10120 tablet_service.cc:1467] Processing CreateTablet for tablet 475784a803d84ffca4ee5c685117841d (DEFAULT_TABLE table=test-table [id=23223d09e9d840b4a62170c302ee5a61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:30.314960 9858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 475784a803d84ffca4ee5c685117841d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:30.315068 10120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 475784a803d84ffca4ee5c685117841d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:30.318410 10201 tablet_bootstrap.cc:492] T 475784a803d84ffca4ee5c685117841d P b56837df97c8422eae538ec66db22116: Bootstrap starting.
I20260925 22:05:30.318955 10198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:30.312453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:43521 (local address 127.0.0.1:45614)
0925 22:05:30.313282 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:30.313290 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:30.313352 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:30.313783 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:30.313784 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:30.313785 (+ 1us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:30.318581 (+ 4796us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:05:30.318582 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:30.318589 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:05:30.318603 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:30.318829 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:30.318830 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:30.318831 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 22:05:30.318876 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 22:05:30.318890 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":813,"thread_start_us":66,"threads_started":1}
I20260925 22:05:30.318975 10199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:30.312548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45614 (local address 127.5.0.66:43521)
0925 22:05:30.312735 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:30.312739 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:30.313406 (+ 667us) server_negotiation.cc:407] Connection header received
0925 22:05:30.313435 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:30.313437 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:30.313447 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:30.313515 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:30.318672 (+ 5157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:30.318679 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:30.318679 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:05:30.318683 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:30.318725 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:05:30.318754 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:30.318823 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:30.318882 (+ 59us) server_negotiation.cc:299] Negotiation successful
0925 22:05:30.318898 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":72,"threads_started":1}
I20260925 22:05:30.318991 10201 tablet_bootstrap.cc:654] T 475784a803d84ffca4ee5c685117841d P b56837df97c8422eae538ec66db22116: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:30.319326 10201 log.cc:824] T 475784a803d84ffca4ee5c685117841d P b56837df97c8422eae538ec66db22116: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:30.319633 10200 tablet_bootstrap.cc:492] T 475784a803d84ffca4ee5c685117841d P 94e5320e6f16477cadc38c2c19443143: Bootstrap starting.
I20260925 22:05:30.319597 9989 tablet_service.cc:1467] Processing CreateTablet for tablet 475784a803d84ffca4ee5c685117841d (DEFAULT_TABLE table=test-table [id=23223d09e9d840b4a62170c302ee5a61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:30.319953 9989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 475784a803d84ffca4ee5c685117841d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:30.320123 10201 tablet_bootstrap.cc:492] T 475784a803d84ffca4ee5c685117841d P b56837df97c8422eae538ec66db22116: No bootstrap required, opened a new log
I20260925 22:05:30.320190 10201 ts_tablet_manager.cc:1397] T 475784a803d84ffca4ee5c685117841d P b56837df97c8422eae538ec66db22116: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:05:30.320289 10200 tablet_bootstrap.cc:654] T 475784a803d84ffca4ee5c685117841d P 94e5320e6f16477cadc38c2c19443143: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:30.320554 10200 log.cc:824] T 475784a803d84ffca4ee5c685117841d P 94e5320e6f16477cadc38c2c19443143: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:30.321199 10200 tablet_bootstrap.cc:492] T 475784a803d84ffca4ee5c685117841d P 94e5320e6f16477cadc38c2c19443143: No bootstrap required, opened a new log
I20260925 22:05:30.321264 10200 ts_tablet_manager.cc:1397] T 475784a803d84ffca4ee5c685117841d P 94e5320e6f16477cadc38c2c19443143: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:05:30.322702 10201 raft_consensus.cc:348] T 475784a803d84ffca4ee5c685117841d P b56837df97c8422eae538ec66db22116 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b56837df97c8422eae538ec66db22116" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 39733 } } peers { permanent_uuid: "94e5320e6f16477cadc38c2c19443143" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 43807 } } peers { permanent_uuid: "32dcd0b6f3384455a4d1ea3a77781c6c" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 43521 } }
I20260925 22:05:30.322882 10201 raft_consensus.cc:374] T 475784a803d84ffca4ee5c685117841d P b56837df97c8422eae538ec66db22116 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:30.322921 10201 raft_consensus.cc:729] T 475784a803d84ffca4ee5c685117841d P b56837df97c8422eae538ec66db22116 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b56837df97c8422eae538ec66db22116, State: Initialized, Role: FOLLOWER
I20260925 22:05:30.323055 10201 consensus_queue.cc:260] T 475784a803d84ffca4ee5c685117841d P b56837df97c8422eae538ec66db22116 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b56837df97c8422eae538ec66db22116" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 39733 } } peers { permanent_uuid: "94e5320e6f16477cadc38c2c19443143" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 43807 } } peers { permanent_uuid: "32dcd0b6f3384455a4d1ea3a77781c6c" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 43521 } }
I20260925 22:05:30.323107 10200 raft_consensus.cc:348] T 475784a803d84ffca4ee5c685117841d P 94e5320e6f16477cadc38c2c19443143 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b56837df97c8422eae538ec66db22116" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 39733 } } peers { permanent_uuid: "94e5320e6f16477cadc38c2c19443143" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 43807 } } peers { permanent_uuid: "32dcd0b6f3384455a4d1ea3a77781c6c" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 43521 } }
I20260925 22:05:30.323292 10200 raft_consensus.cc:374] T 475784a803d84ffca4ee5c685117841d P 94e5320e6f16477cadc38c2c19443143 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:30.323338 10200 raft_consensus.cc:729] T 475784a803d84ffca4ee5c685117841d P 94e5320e6f16477cadc38c2c19443143 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94e5320e6f16477cadc38c2c19443143, State: Initialized, Role: FOLLOWER
I20260925 22:05:30.323446 9924 heartbeater.cc:503] Master 127.5.0.126:46765 was elected leader, sending a full tablet report...
I20260925 22:05:30.323455 10204 tablet_bootstrap.cc:492] T 475784a803d84ffca4ee5c685117841d P 32dcd0b6f3384455a4d1ea3a77781c6c: Bootstrap starting.
I20260925 22:05:30.323477 10200 consensus_queue.cc:260] T 475784a803d84ffca4ee5c685117841d P 94e5320e6f16477cadc38c2c19443143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b56837df97c8422eae538ec66db22116" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 39733 } } peers { permanent_uuid: "94e5320e6f16477cadc38c2c19443143" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 43807 } } peers { permanent_uuid: "32dcd0b6f3384455a4d1ea3a77781c6c" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 43521 } }
I20260925 22:05:30.323757 10186 heartbeater.cc:503] Master 127.5.0.126:46765 was elected leader, sending a full tablet report...
I20260925 22:05:30.323446 10201 ts_tablet_manager.cc:1428] T 475784a803d84ffca4ee5c685117841d P b56837df97c8422eae538ec66db22116: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260925 22:05:30.323741 10200 ts_tablet_manager.cc:1428] T 475784a803d84ffca4ee5c685117841d P 94e5320e6f16477cadc38c2c19443143: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 22:05:30.324117 10204 tablet_bootstrap.cc:654] T 475784a803d84ffca4ee5c685117841d P 32dcd0b6f3384455a4d1ea3a77781c6c: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:30.324429 10204 log.cc:824] T 475784a803d84ffca4ee5c685117841d P 32dcd0b6f3384455a4d1ea3a77781c6c: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:30.325161 10204 tablet_bootstrap.cc:492] T 475784a803d84ffca4ee5c685117841d P 32dcd0b6f3384455a4d1ea3a77781c6c: No bootstrap required, opened a new log
I20260925 22:05:30.325215 10204 ts_tablet_manager.cc:1397] T 475784a803d84ffca4ee5c685117841d P 32dcd0b6f3384455a4d1ea3a77781c6c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:05:30.326877 10204 raft_consensus.cc:348] T 475784a803d84ffca4ee5c685117841d P 32dcd0b6f3384455a4d1ea3a77781c6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b56837df97c8422eae538ec66db22116" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 39733 } } peers { permanent_uuid: "94e5320e6f16477cadc38c2c19443143" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 43807 } } peers { permanent_uuid: "32dcd0b6f3384455a4d1ea3a77781c6c" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 43521 } }
I20260925 22:05:30.326992 10204 raft_consensus.cc:374] T 475784a803d84ffca4ee5c685117841d P 32dcd0b6f3384455a4d1ea3a77781c6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:30.327013 10204 raft_consensus.cc:729] T 475784a803d84ffca4ee5c685117841d P 32dcd0b6f3384455a4d1ea3a77781c6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32dcd0b6f3384455a4d1ea3a77781c6c, State: Initialized, Role: FOLLOWER
I20260925 22:05:30.327116 10204 consensus_queue.cc:260] T 475784a803d84ffca4ee5c685117841d P 32dcd0b6f3384455a4d1ea3a77781c6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b56837df97c8422eae538ec66db22116" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 39733 } } peers { permanent_uuid: "94e5320e6f16477cadc38c2c19443143" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 43807 } } peers { permanent_uuid: "32dcd0b6f3384455a4d1ea3a77781c6c" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 43521 } }
I20260925 22:05:30.327325 10204 ts_tablet_manager.cc:1428] T 475784a803d84ffca4ee5c685117841d P 32dcd0b6f3384455a4d1ea3a77781c6c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:05:30.327387 10055 heartbeater.cc:503] Master 127.5.0.126:46765 was elected leader, sending a full tablet report...
W20260925 22:05:30.435585 10056 tablet.cc:2369] T 475784a803d84ffca4ee5c685117841d P 32dcd0b6f3384455a4d1ea3a77781c6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:05:30.523669 10206 raft_consensus.cc:482] T 475784a803d84ffca4ee5c685117841d P 94e5320e6f16477cadc38c2c19443143 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:05:30.523823 10206 raft_consensus.cc:504] T 475784a803d84ffca4ee5c685117841d P 94e5320e6f16477cadc38c2c19443143 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b56837df97c8422eae538ec66db22116" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 39733 } } peers { permanent_uuid: "94e5320e6f16477cadc38c2c19443143" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 43807 } } peers { permanent_uuid: "32dcd0b6f3384455a4d1ea3a77781c6c" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 43521 } }
I20260925 22:05:30.524204 10206 leader_election.cc:288] T 475784a803d84ffca4ee5c685117841d P 94e5320e6f16477cadc38c2c19443143 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b56837df97c8422eae538ec66db22116 (127.5.0.65:39733), 32dcd0b6f3384455a4d1ea3a77781c6c (127.5.0.66:43521)
I20260925 22:05:30.525547 10188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:30.524398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:39733 (local address 127.5.0.67:49627)
0925 22:05:30.524455 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:30.524469 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:30.524510 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:30.524679 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:30.524680 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:30.524683 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:30.525214 (+ 531us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:05:30.525215 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:30.525221 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:05:30.525232 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:30.525401 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:30.525401 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:30.525402 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 22:05:30.525451 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 22:05:30.525466 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260925 22:05:30.525578 10196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:30.524474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:49627 (local address 127.5.0.65:39733)
0925 22:05:30.524528 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:30.524530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:30.524546 (+ 16us) server_negotiation.cc:407] Connection header received
0925 22:05:30.524571 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:30.524572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:30.524576 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:30.524664 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:30.525307 (+ 643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:30.525308 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:30.525308 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:05:30.525312 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:30.525327 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:05:30.525331 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:30.525392 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:30.525480 (+ 88us) server_negotiation.cc:299] Negotiation successful
0925 22:05:30.525497 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 22:05:30.525693 10209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:30.524525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:43521 (local address 127.5.0.67:35917)
0925 22:05:30.524777 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:30.524785 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:30.524827 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:30.524960 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:30.524960 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:30.524963 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:30.525367 (+ 404us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:05:30.525368 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:30.525375 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:05:30.525387 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:30.525550 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:30.525551 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:30.525552 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 22:05:30.525592 (+ 40us) client_negotiation.cc:240] Negotiation successful
0925 22:05:30.525605 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":96,"threads_started":1}
I20260925 22:05:30.525704 10199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:30.524680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:35917 (local address 127.5.0.66:43521)
0925 22:05:30.524741 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:30.524743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:30.524839 (+ 96us) server_negotiation.cc:407] Connection header received
0925 22:05:30.524901 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:30.524902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:30.524905 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:30.524948 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:30.525479 (+ 531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:30.525479 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:30.525480 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:05:30.525484 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:30.525497 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:05:30.525501 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:30.525552 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:30.525601 (+ 49us) server_negotiation.cc:299] Negotiation successful
0925 22:05:30.525613 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 22:05:30.525882 9878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "475784a803d84ffca4ee5c685117841d" candidate_uuid: "94e5320e6f16477cadc38c2c19443143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b56837df97c8422eae538ec66db22116" is_pre_election: true
I20260925 22:05:30.525996 9878 raft_consensus.cc:2454] T 475784a803d84ffca4ee5c685117841d P b56837df97c8422eae538ec66db22116 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94e5320e6f16477cadc38c2c19443143 in term 0.
I20260925 22:05:30.526213 10076 leader_election.cc:302] T 475784a803d84ffca4ee5c685117841d P 94e5320e6f16477cadc38c2c19443143 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94e5320e6f16477cadc38c2c19443143, b56837df97c8422eae538ec66db22116; no voters:
I20260925 22:05:30.526353 10206 raft_consensus.cc:2781] T 475784a803d84ffca4ee5c685117841d P 94e5320e6f16477cadc38c2c19443143 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:05:30.526420 10206 raft_consensus.cc:482] T 475784a803d84ffca4ee5c685117841d P 94e5320e6f16477cadc38c2c19443143 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:05:30.526446 10206 raft_consensus.cc:3037] T 475784a803d84ffca4ee5c685117841d P 94e5320e6f16477cadc38c2c19443143 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:30.526546 10009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "475784a803d84ffca4ee5c685117841d" candidate_uuid: "94e5320e6f16477cadc38c2c19443143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32dcd0b6f3384455a4d1ea3a77781c6c" is_pre_election: true
I20260925 22:05:30.526693 10009 raft_consensus.cc:2454] T 475784a803d84ffca4ee5c685117841d P 32dcd0b6f3384455a4d1ea3a77781c6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94e5320e6f16477cadc38c2c19443143 in term 0.
I20260925 22:05:30.527242 10206 raft_consensus.cc:504] T 475784a803d84ffca4ee5c685117841d P 94e5320e6f16477cadc38c2c19443143 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b56837df97c8422eae538ec66db22116" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 39733 } } peers { permanent_uuid: "94e5320e6f16477cadc38c2c19443143" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 43807 } } peers { permanent_uuid: "32dcd0b6f3384455a4d1ea3a77781c6c" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 43521 } }
I20260925 22:05:30.527369 10206 leader_election.cc:288] T 475784a803d84ffca4ee5c685117841d P 94e5320e6f16477cadc38c2c19443143 [CANDIDATE]: Term 1 election: Requested vote from peers b56837df97c8422eae538ec66db22116 (127.5.0.65:39733), 32dcd0b6f3384455a4d1ea3a77781c6c (127.5.0.66:43521)
I20260925 22:05:30.527506 9878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "475784a803d84ffca4ee5c685117841d" candidate_uuid: "94e5320e6f16477cadc38c2c19443143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b56837df97c8422eae538ec66db22116"
I20260925 22:05:30.527531 10009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "475784a803d84ffca4ee5c685117841d" candidate_uuid: "94e5320e6f16477cadc38c2c19443143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32dcd0b6f3384455a4d1ea3a77781c6c"
I20260925 22:05:30.527591 9878 raft_consensus.cc:3037] T 475784a803d84ffca4ee5c685117841d P b56837df97c8422eae538ec66db22116 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:30.527597 10009 raft_consensus.cc:3037] T 475784a803d84ffca4ee5c685117841d P 32dcd0b6f3384455a4d1ea3a77781c6c [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:30.528331 9878 raft_consensus.cc:2454] T 475784a803d84ffca4ee5c685117841d P b56837df97c8422eae538ec66db22116 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94e5320e6f16477cadc38c2c19443143 in term 1.
I20260925 22:05:30.528388 10009 raft_consensus.cc:2454] T 475784a803d84ffca4ee5c685117841d P 32dcd0b6f3384455a4d1ea3a77781c6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94e5320e6f16477cadc38c2c19443143 in term 1.
I20260925 22:05:30.528512 10076 leader_election.cc:302] T 475784a803d84ffca4ee5c685117841d P 94e5320e6f16477cadc38c2c19443143 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94e5320e6f16477cadc38c2c19443143, b56837df97c8422eae538ec66db22116; no voters:
I20260925 22:05:30.528589 10206 raft_consensus.cc:2781] T 475784a803d84ffca4ee5c685117841d P 94e5320e6f16477cadc38c2c19443143 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:30.528762 10206 raft_consensus.cc:686] T 475784a803d84ffca4ee5c685117841d P 94e5320e6f16477cadc38c2c19443143 [term 1 LEADER]: Becoming Leader. State: Replica: 94e5320e6f16477cadc38c2c19443143, State: Running, Role: LEADER
I20260925 22:05:30.528855 10206 consensus_queue.cc:237] T 475784a803d84ffca4ee5c685117841d P 94e5320e6f16477cadc38c2c19443143 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b56837df97c8422eae538ec66db22116" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 39733 } } peers { permanent_uuid: "94e5320e6f16477cadc38c2c19443143" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 43807 } } peers { permanent_uuid: "32dcd0b6f3384455a4d1ea3a77781c6c" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 43521 } }
I20260925 22:05:30.529608 9736 catalog_manager.cc:5495] T 475784a803d84ffca4ee5c685117841d P 94e5320e6f16477cadc38c2c19443143 reported cstate change: term changed from 0 to 1, leader changed from <none> to 94e5320e6f16477cadc38c2c19443143 (127.5.0.67). New cstate: current_term: 1 leader_uuid: "94e5320e6f16477cadc38c2c19443143" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b56837df97c8422eae538ec66db22116" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 39733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94e5320e6f16477cadc38c2c19443143" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 43807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32dcd0b6f3384455a4d1ea3a77781c6c" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 43521 } health_report { overall_health: UNKNOWN } } }
I20260925 22:05:30.541045 10197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:30.540024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39434 (local address 127.5.0.67:43807)
0925 22:05:30.540083 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:30.540086 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:30.540124 (+ 38us) server_negotiation.cc:407] Connection header received
0925 22:05:30.540214 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:30.540215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:30.540235 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:30.540297 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:30.540812 (+ 515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:30.540813 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:30.540813 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:05:30.540818 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:30.540832 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:05:30.540836 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:30.540896 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:30.540951 (+ 55us) server_negotiation.cc:299] Negotiation successful
0925 22:05:30.540967 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 22:05:30.542878 9878 raft_consensus.cc:1261] T 475784a803d84ffca4ee5c685117841d P b56837df97c8422eae538ec66db22116 [term 1 FOLLOWER]: Refusing update from remote peer 94e5320e6f16477cadc38c2c19443143: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:05:30.542878 10009 raft_consensus.cc:1261] T 475784a803d84ffca4ee5c685117841d P 32dcd0b6f3384455a4d1ea3a77781c6c [term 1 FOLLOWER]: Refusing update from remote peer 94e5320e6f16477cadc38c2c19443143: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:05:30.543223 10206 consensus_queue.cc:1035] T 475784a803d84ffca4ee5c685117841d P 94e5320e6f16477cadc38c2c19443143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32dcd0b6f3384455a4d1ea3a77781c6c" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 43521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:05:30.543370 10210 consensus_queue.cc:1035] T 475784a803d84ffca4ee5c685117841d P 94e5320e6f16477cadc38c2c19443143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b56837df97c8422eae538ec66db22116" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 39733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:05:30.546298 10214 mvcc.cc:205] Tried to move back new op lower bound from 7333371619500716032 to 7333371619446755328. Current Snapshot: MvccSnapshot[applied={T|T < 7333371619500716032}]
I20260925 22:05:30.546526 10215 mvcc.cc:205] Tried to move back new op lower bound from 7333371619500716032 to 7333371619446755328. Current Snapshot: MvccSnapshot[applied={T|T < 7333371619500716032 or (T in {7333371619500716032})}]
I20260925 22:05:30.547796 9736 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:05:30.547914 9736 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:05:30.548625 9736 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 44782d782dbd4db89c732b3d12bfa5ce: Sending DeleteTablet for 3 replicas of tablet 475784a803d84ffca4ee5c685117841d
I20260925 22:05:30.548947 9858 tablet_service.cc:1514] Processing DeleteTablet for tablet 475784a803d84ffca4ee5c685117841d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:30 UTC) from {username='slave'} at 127.0.0.1:46498
I20260925 22:05:30.549009 10120 tablet_service.cc:1514] Processing DeleteTablet for tablet 475784a803d84ffca4ee5c685117841d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:30 UTC) from {username='slave'} at 127.0.0.1:39426
I20260925 22:05:30.549237 9989 tablet_service.cc:1514] Processing DeleteTablet for tablet 475784a803d84ffca4ee5c685117841d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:30 UTC) from {username='slave'} at 127.0.0.1:45614
I20260925 22:05:30.549273 10222 tablet_replica.cc:330] T 475784a803d84ffca4ee5c685117841d P b56837df97c8422eae538ec66db22116: stopping tablet replica
I20260925 22:05:30.549364 10222 raft_consensus.cc:2229] T 475784a803d84ffca4ee5c685117841d P b56837df97c8422eae538ec66db22116 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:05:30.549450 10223 tablet_replica.cc:330] T 475784a803d84ffca4ee5c685117841d P 94e5320e6f16477cadc38c2c19443143: stopping tablet replica
I20260925 22:05:30.549470 10222 raft_consensus.cc:2258] T 475784a803d84ffca4ee5c685117841d P b56837df97c8422eae538ec66db22116 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:30.549525 10223 raft_consensus.cc:2229] T 475784a803d84ffca4ee5c685117841d P 94e5320e6f16477cadc38c2c19443143 [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:05:30.549603 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 9796
I20260925 22:05:30.549643 10223 raft_consensus.cc:2258] T 475784a803d84ffca4ee5c685117841d P 94e5320e6f16477cadc38c2c19443143 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:30.549763 10224 tablet_replica.cc:330] T 475784a803d84ffca4ee5c685117841d P 32dcd0b6f3384455a4d1ea3a77781c6c: stopping tablet replica
I20260925 22:05:30.549839 10224 raft_consensus.cc:2229] T 475784a803d84ffca4ee5c685117841d P 32dcd0b6f3384455a4d1ea3a77781c6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:05:30.549913 10224 raft_consensus.cc:2258] T 475784a803d84ffca4ee5c685117841d P 32dcd0b6f3384455a4d1ea3a77781c6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:30.550920 10223 ts_tablet_manager.cc:1905] T 475784a803d84ffca4ee5c685117841d P 94e5320e6f16477cadc38c2c19443143: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:30.552402 10223 ts_tablet_manager.cc:1918] T 475784a803d84ffca4ee5c685117841d P 94e5320e6f16477cadc38c2c19443143: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:05:30.552474 10223 log.cc:1196] T 475784a803d84ffca4ee5c685117841d P 94e5320e6f16477cadc38c2c19443143: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-2/wal/wals/475784a803d84ffca4ee5c685117841d
I20260925 22:05:30.552752 10223 ts_tablet_manager.cc:1939] T 475784a803d84ffca4ee5c685117841d P 94e5320e6f16477cadc38c2c19443143: Deleting consensus metadata
I20260925 22:05:30.553073 10224 ts_tablet_manager.cc:1905] T 475784a803d84ffca4ee5c685117841d P 32dcd0b6f3384455a4d1ea3a77781c6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:30.553448 9720 catalog_manager.cc:4841] TS 94e5320e6f16477cadc38c2c19443143 (127.5.0.67:43807): tablet 475784a803d84ffca4ee5c685117841d (table test-table [id=23223d09e9d840b4a62170c302ee5a61]) successfully deleted
I20260925 22:05:30.554944 10224 ts_tablet_manager.cc:1918] T 475784a803d84ffca4ee5c685117841d P 32dcd0b6f3384455a4d1ea3a77781c6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:05:30.555022 10224 log.cc:1196] T 475784a803d84ffca4ee5c685117841d P 32dcd0b6f3384455a4d1ea3a77781c6c: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373908987421-5121-0/minicluster-data/ts-1/wal/wals/475784a803d84ffca4ee5c685117841d
I20260925 22:05:30.555284 10224 ts_tablet_manager.cc:1939] T 475784a803d84ffca4ee5c685117841d P 32dcd0b6f3384455a4d1ea3a77781c6c: Deleting consensus metadata
I20260925 22:05:30.555852 9723 catalog_manager.cc:4841] TS 32dcd0b6f3384455a4d1ea3a77781c6c (127.5.0.66:43521): tablet 475784a803d84ffca4ee5c685117841d (table test-table [id=23223d09e9d840b4a62170c302ee5a61]) successfully deleted
W20260925 22:05:30.556665 9723 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.0.65:39733 (error 108)
I20260925 22:05:30.556875 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 9927
I20260925 22:05:30.557473 10198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:30.556989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:39733 (local address 127.0.0.1:46512)
0925 22:05:30.557401 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:30.557437 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.65:39733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392}
W20260925 22:05:30.557583 9723 catalog_manager.cc:4584] TS b56837df97c8422eae538ec66db22116 (127.5.0.65:39733): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 475784a803d84ffca4ee5c685117841d: Network error: Client connection negotiation failed: client connection to 127.5.0.65:39733: connect: Connection refused (error 111)
I20260925 22:05:30.563395 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 10058
I20260925 22:05:30.568091 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 9705
2026-09-25T22:05:30Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (779 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260925 22:05:30.587368 5121 mini_oidc.cc:152] Starting JWKS server
I20260925 22:05:30.588503 5121 webserver.cc:460] Webserver started at https://127.0.0.1:40431/ using document root <none> and password file <none>
I20260925 22:05:30.588609 5121 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 22:05:30.588780 5121 webserver.cc:460] Webserver started at http://127.0.0.1:35795/ using document root <none> and password file <none>
2026-09-25T22:05:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:05:30Z Disabled control of system clock
I20260925 22:05:30.596192 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:44537
--webserver_interface=127.5.0.126
--webserver_port=0
--builtin_ntp_servers=127.5.0.84:33423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:44537
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40431/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373908987421-5121-0/testchainca.cert with env {}
W20260925 22:05:30.676242 10231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:30.676453 10231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:30.676476 10231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:30.677690 10231 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:05:30.677735 10231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:30.677748 10231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:30.677762 10231 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:05:30.677814 10231 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 22:05:30.677831 10231 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 22:05:30.677853 10231 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40431/jwks.json
I20260925 22:05:30.679437 10231 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:33423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:44537
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:44537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40431/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373908987421-5121-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:30.679705 10231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:30.679997 10231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:30.685114 10236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:30.685045 10239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:30.685168 10231 server_base.cc:1034] running on GCE node
W20260925 22:05:30.685063 10237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:30.685575 10231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:30.685885 10231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:30.687047 10231 hybrid_clock.cc:648] HybridClock initialized: now 1790373930687034 us; error 32 us; skew 500 ppm
I20260925 22:05:30.688321 10231 webserver.cc:460] Webserver started at http://127.5.0.126:44545/ using document root <none> and password file <none>
I20260925 22:05:30.688575 10231 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:30.688647 10231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:30.688798 10231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:30.689904 10231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "a7d7809e620942cfabef2d13221ced8b"
format_stamp: "Formatted at 2026-09-25 22:05:30 on dist-test-slave-7mqr"
server_key: "3be22b3e6922edbc35d6dd64dd565587"
server_key_iv: "0353984a060562e94920ee3ae21713b0"
server_key_version: "encryptionkey@0"
I20260925 22:05:30.690315 10231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "a7d7809e620942cfabef2d13221ced8b"
format_stamp: "Formatted at 2026-09-25 22:05:30 on dist-test-slave-7mqr"
server_key: "3be22b3e6922edbc35d6dd64dd565587"
server_key_iv: "0353984a060562e94920ee3ae21713b0"
server_key_version: "encryptionkey@0"
I20260925 22:05:30.691638 10231 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260925 22:05:30.692466 10245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:30.692643 10231 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:30.692759 10231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "a7d7809e620942cfabef2d13221ced8b"
format_stamp: "Formatted at 2026-09-25 22:05:30 on dist-test-slave-7mqr"
server_key: "3be22b3e6922edbc35d6dd64dd565587"
server_key_iv: "0353984a060562e94920ee3ae21713b0"
server_key_version: "encryptionkey@0"
I20260925 22:05:30.692843 10231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:30.701647 10231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:30.701988 10231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:30.702112 10231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:30.706785 10231 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:44537
I20260925 22:05:30.706825 10297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:44537 every 8 connection(s)
I20260925 22:05:30.707181 10231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:05:30.708220 10298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:30.710210 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 10231
I20260925 22:05:30.710321 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:05:30.710490 10298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7d7809e620942cfabef2d13221ced8b: Bootstrap starting.
I20260925 22:05:30.710507 5121 external_mini_cluster.cc:1428] Setting key 11c801144308c7961ffcf74ef77c7fad
I20260925 22:05:30.711241 10298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7d7809e620942cfabef2d13221ced8b: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:30.711567 10298 log.cc:824] T 00000000000000000000000000000000 P a7d7809e620942cfabef2d13221ced8b: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:30.712358 10298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7d7809e620942cfabef2d13221ced8b: No bootstrap required, opened a new log
I20260925 22:05:30.714555 10298 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7d7809e620942cfabef2d13221ced8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7d7809e620942cfabef2d13221ced8b" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 44537 } }
I20260925 22:05:30.714723 10298 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7d7809e620942cfabef2d13221ced8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:30.714756 10298 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7d7809e620942cfabef2d13221ced8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7d7809e620942cfabef2d13221ced8b, State: Initialized, Role: FOLLOWER
I20260925 22:05:30.714879 10298 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7d7809e620942cfabef2d13221ced8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7d7809e620942cfabef2d13221ced8b" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 44537 } }
I20260925 22:05:30.714946 10298 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7d7809e620942cfabef2d13221ced8b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:05:30.714974 10298 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7d7809e620942cfabef2d13221ced8b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:05:30.715019 10298 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7d7809e620942cfabef2d13221ced8b [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:30.715749 10298 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7d7809e620942cfabef2d13221ced8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7d7809e620942cfabef2d13221ced8b" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 44537 } }
I20260925 22:05:30.715881 10298 leader_election.cc:302] T 00000000000000000000000000000000 P a7d7809e620942cfabef2d13221ced8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7d7809e620942cfabef2d13221ced8b; no voters:
I20260925 22:05:30.716071 10298 leader_election.cc:288] T 00000000000000000000000000000000 P a7d7809e620942cfabef2d13221ced8b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:05:30.716127 10303 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7d7809e620942cfabef2d13221ced8b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:30.716274 10303 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7d7809e620942cfabef2d13221ced8b [term 1 LEADER]: Becoming Leader. State: Replica: a7d7809e620942cfabef2d13221ced8b, State: Running, Role: LEADER
I20260925 22:05:30.716351 10298 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7d7809e620942cfabef2d13221ced8b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:05:30.716369 10303 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7d7809e620942cfabef2d13221ced8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7d7809e620942cfabef2d13221ced8b" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 44537 } }
I20260925 22:05:30.716637 10303 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7d7809e620942cfabef2d13221ced8b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7d7809e620942cfabef2d13221ced8b. Latest consensus state: current_term: 1 leader_uuid: "a7d7809e620942cfabef2d13221ced8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7d7809e620942cfabef2d13221ced8b" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 44537 } } }
I20260925 22:05:30.716728 10303 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7d7809e620942cfabef2d13221ced8b [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:30.716647 10305 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7d7809e620942cfabef2d13221ced8b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7d7809e620942cfabef2d13221ced8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7d7809e620942cfabef2d13221ced8b" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 44537 } } }
I20260925 22:05:30.716781 10305 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7d7809e620942cfabef2d13221ced8b [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:30.717005 10301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:30.711011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44480 (local address 127.5.0.126:44537)
0925 22:05:30.711241 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:30.711250 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:30.711284 (+ 34us) server_negotiation.cc:407] Connection header received
0925 22:05:30.711858 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:30.711866 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:30.711985 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:30.712145 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:30.712337 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:30.713713 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:30.714462 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:30.715066 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:30.715206 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:30.715826 (+ 620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:30.715830 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:30.715831 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:05:30.716063 (+ 232us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:30.716105 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:05:30.716120 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:30.716198 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:30.716490 (+ 292us) server_negotiation.cc:299] Negotiation successful
0925 22:05:30.716578 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":83,"threads_started":1}
I20260925 22:05:30.717928 5121 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260925 22:05:30.718241 10318 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a7d7809e620942cfabef2d13221ced8b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:05:30.718294 10318 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:05:30.718355 10308 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:05:30.718544 10308 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:05:30.719879 10308 catalog_manager.cc:1347] Generated new cluster ID: 3701fedd8db84249a89b1ddb9c0ee406
I20260925 22:05:30.719951 10308 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:05:30.723714 10308 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:05:30.724244 10308 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:05:30.727656 10308 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7d7809e620942cfabef2d13221ced8b: Generated new TSK 0
I20260925 22:05:30.727821 10308 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260925 22:05:30.749783 10328 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260925 22:05:30.749894 10328 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260925 22:05:30.750221 10301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:30.735246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44496 (local address 127.5.0.126:44537)
0925 22:05:30.735308 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:30.735310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:30.735457 (+ 147us) server_negotiation.cc:407] Connection header received
0925 22:05:30.735597 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:30.735598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:30.735611 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:30.735694 (+ 83us) server_negotiation.cc:226] Negotiated authn=JWT
0925 22:05:30.735864 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:30.736396 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:30.737219 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:30.737421 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:30.737490 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:30.737709 (+ 219us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 22:05:30.749901 (+ 12192us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40431/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0925 22:05:30.750127 (+ 226us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44496: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40431/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":37}
W20260925 22:05:30.750277 10301 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44496: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40431/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260925 22:05:30.750519 10327 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:30.735187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:44537 (local address 127.0.0.1:44496)
0925 22:05:30.735391 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:30.735404 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:30.735494 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:30.735700 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:30.735702 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:30.735704 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0925 22:05:30.735787 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:30.735789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:30.736522 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:30.736524 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:30.737118 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:30.737121 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:30.737508 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:30.737512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:30.737590 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:30.737607 (+ 17us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 22:05:30.750087 (+ 12480us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40431/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0925 22:05:30.750196 (+ 109us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.0.126:44537: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40431/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":110,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.5.0.126:44537: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40431/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260925 22:05:30.751009 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 10231
2026-09-25T22:05:30Z chronyd exiting
I20260925 22:05:30.786849 5121 test_util.cc:175] -----------------------------------------------
I20260925 22:05:30.786949 5121 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373908987421-5121-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (200 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260925 22:05:30.787645 5121 mini_oidc.cc:152] Starting JWKS server
I20260925 22:05:30.788379 5121 webserver.cc:460] Webserver started at https://127.0.0.1:38607/ using document root <none> and password file <none>
I20260925 22:05:30.788442 5121 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 22:05:30.788599 5121 webserver.cc:460] Webserver started at http://127.0.0.1:41699/ using document root <none> and password file <none>
2026-09-25T22:05:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:05:30Z Disabled control of system clock
I20260925 22:05:30.796310 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:43593
--webserver_interface=127.5.0.126
--webserver_port=0
--builtin_ntp_servers=127.5.0.84:34843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:43593
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38607/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373908987421-5121-0/testchainca.cert with env {}
W20260925 22:05:30.875502 10334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:30.875717 10334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:30.875741 10334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:30.876967 10334 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:05:30.877012 10334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:30.877025 10334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:30.877061 10334 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:05:30.877089 10334 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 22:05:30.877112 10334 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 22:05:30.877137 10334 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38607/jwks.json
I20260925 22:05:30.878633 10334 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:34843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:43593
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:43593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38607/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373908987421-5121-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:30.878865 10334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:30.879065 10334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:30.883764 10340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:30.883862 10342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:30.883934 10339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:30.884019 10334 server_base.cc:1034] running on GCE node
I20260925 22:05:30.884235 10334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:30.884478 10334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:30.885619 10334 hybrid_clock.cc:648] HybridClock initialized: now 1790373930885605 us; error 30 us; skew 500 ppm
I20260925 22:05:30.886848 10334 webserver.cc:460] Webserver started at http://127.5.0.126:44259/ using document root <none> and password file <none>
I20260925 22:05:30.887089 10334 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:30.887136 10334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:30.887298 10334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:30.888404 10334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "0cb541f996664aefbf4b43267d5378de"
format_stamp: "Formatted at 2026-09-25 22:05:30 on dist-test-slave-7mqr"
server_key: "72f77e251540534ddf830ed937362908"
server_key_iv: "206dd58a851d50894e66409034f9dd4f"
server_key_version: "encryptionkey@0"
I20260925 22:05:30.888761 10334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "0cb541f996664aefbf4b43267d5378de"
format_stamp: "Formatted at 2026-09-25 22:05:30 on dist-test-slave-7mqr"
server_key: "72f77e251540534ddf830ed937362908"
server_key_iv: "206dd58a851d50894e66409034f9dd4f"
server_key_version: "encryptionkey@0"
I20260925 22:05:30.890043 10334 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:05:30.890710 10348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:30.890875 10334 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:05:30.890975 10334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "0cb541f996664aefbf4b43267d5378de"
format_stamp: "Formatted at 2026-09-25 22:05:30 on dist-test-slave-7mqr"
server_key: "72f77e251540534ddf830ed937362908"
server_key_iv: "206dd58a851d50894e66409034f9dd4f"
server_key_version: "encryptionkey@0"
I20260925 22:05:30.891045 10334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:30.908339 10334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:30.908646 10334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:30.908793 10334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:30.912671 10334 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:43593
I20260925 22:05:30.912705 10400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:43593 every 8 connection(s)
I20260925 22:05:30.913036 10334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:05:30.913925 10401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:30.916252 10401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cb541f996664aefbf4b43267d5378de: Bootstrap starting.
I20260925 22:05:30.916819 10401 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0cb541f996664aefbf4b43267d5378de: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:30.917100 10401 log.cc:824] T 00000000000000000000000000000000 P 0cb541f996664aefbf4b43267d5378de: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:30.917760 10401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cb541f996664aefbf4b43267d5378de: No bootstrap required, opened a new log
I20260925 22:05:30.919310 10401 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0cb541f996664aefbf4b43267d5378de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cb541f996664aefbf4b43267d5378de" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43593 } }
I20260925 22:05:30.919447 10401 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0cb541f996664aefbf4b43267d5378de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:30.919492 10401 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0cb541f996664aefbf4b43267d5378de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cb541f996664aefbf4b43267d5378de, State: Initialized, Role: FOLLOWER
I20260925 22:05:30.919636 10401 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0cb541f996664aefbf4b43267d5378de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cb541f996664aefbf4b43267d5378de" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43593 } }
I20260925 22:05:30.919696 10401 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0cb541f996664aefbf4b43267d5378de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:05:30.919754 10401 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cb541f996664aefbf4b43267d5378de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:05:30.919806 10401 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0cb541f996664aefbf4b43267d5378de [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:30.920363 10401 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cb541f996664aefbf4b43267d5378de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cb541f996664aefbf4b43267d5378de" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43593 } }
I20260925 22:05:30.920482 10401 leader_election.cc:302] T 00000000000000000000000000000000 P 0cb541f996664aefbf4b43267d5378de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0cb541f996664aefbf4b43267d5378de; no voters:
I20260925 22:05:30.920495 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 10334
I20260925 22:05:30.920611 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:05:30.920675 10401 leader_election.cc:288] T 00000000000000000000000000000000 P 0cb541f996664aefbf4b43267d5378de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:05:30.920755 10404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0cb541f996664aefbf4b43267d5378de [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:30.920833 5121 external_mini_cluster.cc:1428] Setting key 58dd540f3f6a7967f5a924f31d1c0322
I20260925 22:05:30.920955 10401 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0cb541f996664aefbf4b43267d5378de [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:05:30.920979 10404 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0cb541f996664aefbf4b43267d5378de [term 1 LEADER]: Becoming Leader. State: Replica: 0cb541f996664aefbf4b43267d5378de, State: Running, Role: LEADER
I20260925 22:05:30.921060 10404 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0cb541f996664aefbf4b43267d5378de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cb541f996664aefbf4b43267d5378de" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43593 } }
I20260925 22:05:30.921373 10404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cb541f996664aefbf4b43267d5378de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0cb541f996664aefbf4b43267d5378de. Latest consensus state: current_term: 1 leader_uuid: "0cb541f996664aefbf4b43267d5378de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cb541f996664aefbf4b43267d5378de" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43593 } } }
I20260925 22:05:30.921473 10404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cb541f996664aefbf4b43267d5378de [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:30.921386 10405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cb541f996664aefbf4b43267d5378de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0cb541f996664aefbf4b43267d5378de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cb541f996664aefbf4b43267d5378de" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43593 } } }
I20260925 22:05:30.921643 10405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cb541f996664aefbf4b43267d5378de [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:30.921880 10413 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:05:30.922472 10413 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:05:30.924077 10413 catalog_manager.cc:1347] Generated new cluster ID: b7a656196fb84b09b217b6ca3a5e525b
I20260925 22:05:30.924131 10413 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:05:30.927578 10412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:30.921471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45298 (local address 127.5.0.126:43593)
0925 22:05:30.921699 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:30.921721 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:30.921943 (+ 222us) server_negotiation.cc:407] Connection header received
0925 22:05:30.922091 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:30.922100 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:30.922188 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:30.922580 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:30.922673 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:30.924175 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:30.925279 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:30.925939 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:30.926080 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:30.926823 (+ 743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:30.926827 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:30.926827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:05:30.927054 (+ 227us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:30.927102 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:05:30.927117 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:30.927186 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:30.927269 (+ 83us) server_negotiation.cc:299] Negotiation successful
0925 22:05:30.927356 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":97,"threads_started":1}
I20260925 22:05:30.928546 5121 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:05:30.929457 10413 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:05:30.929944 10413 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:05:30.934834 10413 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0cb541f996664aefbf4b43267d5378de: Generated new TSK 0
I20260925 22:05:30.935014 10413 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260925 22:05:30.960170 10431 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260925 22:05:30.960266 10431 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260925 22:05:30.960613 10412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:30.945786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45306 (local address 127.5.0.126:43593)
0925 22:05:30.945852 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:30.945855 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:30.945979 (+ 124us) server_negotiation.cc:407] Connection header received
0925 22:05:30.946035 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:30.946036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:30.946048 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:30.946117 (+ 69us) server_negotiation.cc:226] Negotiated authn=JWT
0925 22:05:30.946243 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:30.947050 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:30.947802 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:30.948165 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:30.948230 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:30.948379 (+ 149us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 22:05:30.960383 (+ 12004us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38607/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0925 22:05:30.960540 (+ 157us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45306: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38607/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":40}
W20260925 22:05:30.960677 10412 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45306: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38607/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260925 22:05:30.960662 10430 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:30.945723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:43593 (local address 127.0.0.1:45306)
0925 22:05:30.945900 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:30.945914 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:30.945968 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:30.946121 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:30.946122 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:30.946124 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0925 22:05:30.946174 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:30.946175 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:30.947149 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:30.947150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:30.947727 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:30.947729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:30.948246 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:30.948249 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:30.948289 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:30.948294 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 22:05:30.960495 (+ 12201us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38607/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0925 22:05:30.960594 (+ 99us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.0.126:43593: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38607/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":96,"threads_started":1}
I20260925 22:05:30.961053 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 10334
2026-09-25T22:05:30Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (210 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260925 22:05:30.998505 5121 mini_oidc.cc:152] Starting JWKS server
I20260925 22:05:30.999205 5121 webserver.cc:460] Webserver started at https://127.0.0.1:39119/ using document root <none> and password file <none>
I20260925 22:05:30.999250 5121 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 22:05:30.999408 5121 webserver.cc:460] Webserver started at http://127.0.0.1:34671/ using document root <none> and password file <none>
2026-09-25T22:05:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:05:31Z Disabled control of system clock
I20260925 22:05:31.007014 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:36045
--webserver_interface=127.5.0.126
--webserver_port=0
--builtin_ntp_servers=127.5.0.84:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:36045
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39119/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260925 22:05:31.088980 10437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:31.089315 10437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:31.089362 10437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:31.090672 10437 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:05:31.090741 10437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:31.090763 10437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:31.090785 10437 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:05:31.090819 10437 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 22:05:31.090842 10437 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 22:05:31.090868 10437 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39119/jwks.json
I20260925 22:05:31.092414 10437 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:36045
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:36045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39119/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:31.092716 10437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:31.093032 10437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:31.098166 10445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:31.098140 10443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:31.098155 10442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:31.098338 10437 server_base.cc:1034] running on GCE node
I20260925 22:05:31.098560 10437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:31.098845 10437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:31.100003 10437 hybrid_clock.cc:648] HybridClock initialized: now 1790373931099991 us; error 27 us; skew 500 ppm
I20260925 22:05:31.101140 10437 webserver.cc:460] Webserver started at http://127.5.0.126:35625/ using document root <none> and password file <none>
I20260925 22:05:31.101378 10437 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:31.101441 10437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:31.101577 10437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:31.102600 10437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "42c71141edae4cbd9faf4f38e726ccf2"
format_stamp: "Formatted at 2026-09-25 22:05:31 on dist-test-slave-7mqr"
server_key: "04abc9b1b33b4c78c311db37c7de23ef"
server_key_iv: "995a02d76399bf711c94b5ff1ddd374b"
server_key_version: "encryptionkey@0"
I20260925 22:05:31.102967 10437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "42c71141edae4cbd9faf4f38e726ccf2"
format_stamp: "Formatted at 2026-09-25 22:05:31 on dist-test-slave-7mqr"
server_key: "04abc9b1b33b4c78c311db37c7de23ef"
server_key_iv: "995a02d76399bf711c94b5ff1ddd374b"
server_key_version: "encryptionkey@0"
I20260925 22:05:31.104297 10437 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:05:31.105012 10451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:31.105185 10437 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:05:31.105276 10437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "42c71141edae4cbd9faf4f38e726ccf2"
format_stamp: "Formatted at 2026-09-25 22:05:31 on dist-test-slave-7mqr"
server_key: "04abc9b1b33b4c78c311db37c7de23ef"
server_key_iv: "995a02d76399bf711c94b5ff1ddd374b"
server_key_version: "encryptionkey@0"
I20260925 22:05:31.105355 10437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:31.113059 10437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:31.113350 10437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:31.113489 10437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:31.117511 10437 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:36045
I20260925 22:05:31.117555 10503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:36045 every 8 connection(s)
I20260925 22:05:31.117902 10437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:05:31.118561 10504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:31.121019 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 10437
I20260925 22:05:31.121069 10504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42c71141edae4cbd9faf4f38e726ccf2: Bootstrap starting.
I20260925 22:05:31.121129 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:05:31.121294 5121 external_mini_cluster.cc:1428] Setting key 2e81e39b99116652e93bf11dedf409c5
I20260925 22:05:31.121747 10504 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42c71141edae4cbd9faf4f38e726ccf2: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:31.122078 10504 log.cc:824] T 00000000000000000000000000000000 P 42c71141edae4cbd9faf4f38e726ccf2: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:31.122864 10504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42c71141edae4cbd9faf4f38e726ccf2: No bootstrap required, opened a new log
I20260925 22:05:31.124918 10504 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42c71141edae4cbd9faf4f38e726ccf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c71141edae4cbd9faf4f38e726ccf2" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 36045 } }
I20260925 22:05:31.125105 10504 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42c71141edae4cbd9faf4f38e726ccf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:31.125152 10504 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42c71141edae4cbd9faf4f38e726ccf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42c71141edae4cbd9faf4f38e726ccf2, State: Initialized, Role: FOLLOWER
I20260925 22:05:31.125285 10504 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42c71141edae4cbd9faf4f38e726ccf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c71141edae4cbd9faf4f38e726ccf2" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 36045 } }
I20260925 22:05:31.125375 10504 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42c71141edae4cbd9faf4f38e726ccf2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:05:31.125417 10504 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42c71141edae4cbd9faf4f38e726ccf2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:05:31.125464 10504 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42c71141edae4cbd9faf4f38e726ccf2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:31.126171 10504 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42c71141edae4cbd9faf4f38e726ccf2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c71141edae4cbd9faf4f38e726ccf2" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 36045 } }
I20260925 22:05:31.126291 10504 leader_election.cc:302] T 00000000000000000000000000000000 P 42c71141edae4cbd9faf4f38e726ccf2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42c71141edae4cbd9faf4f38e726ccf2; no voters:
I20260925 22:05:31.126451 10504 leader_election.cc:288] T 00000000000000000000000000000000 P 42c71141edae4cbd9faf4f38e726ccf2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:05:31.126487 10509 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42c71141edae4cbd9faf4f38e726ccf2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:31.126592 10509 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42c71141edae4cbd9faf4f38e726ccf2 [term 1 LEADER]: Becoming Leader. State: Replica: 42c71141edae4cbd9faf4f38e726ccf2, State: Running, Role: LEADER
I20260925 22:05:31.126674 10509 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42c71141edae4cbd9faf4f38e726ccf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c71141edae4cbd9faf4f38e726ccf2" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 36045 } }
I20260925 22:05:31.126742 10504 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42c71141edae4cbd9faf4f38e726ccf2 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:05:31.127008 10509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42c71141edae4cbd9faf4f38e726ccf2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42c71141edae4cbd9faf4f38e726ccf2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c71141edae4cbd9faf4f38e726ccf2" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 36045 } } }
I20260925 22:05:31.127091 10509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42c71141edae4cbd9faf4f38e726ccf2 [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:31.127223 10507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:31.121783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59636 (local address 127.5.0.126:36045)
0925 22:05:31.121959 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:31.121966 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:31.121991 (+ 25us) server_negotiation.cc:407] Connection header received
0925 22:05:31.122417 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:31.122424 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:31.122480 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:31.122598 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:31.122846 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:31.123769 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:31.125055 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:31.125639 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:31.125736 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:31.126406 (+ 670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:31.126410 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:31.126410 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:05:31.126626 (+ 216us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:31.126681 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:05:31.126696 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:31.126782 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:31.126876 (+ 94us) server_negotiation.cc:299] Negotiation successful
0925 22:05:31.126955 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":75,"threads_started":1}
I20260925 22:05:31.127247 10509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42c71141edae4cbd9faf4f38e726ccf2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42c71141edae4cbd9faf4f38e726ccf2. Latest consensus state: current_term: 1 leader_uuid: "42c71141edae4cbd9faf4f38e726ccf2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c71141edae4cbd9faf4f38e726ccf2" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 36045 } } }
I20260925 22:05:31.127338 10509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42c71141edae4cbd9faf4f38e726ccf2 [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:31.127514 10514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:05:31.128036 10514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:05:31.128193 5121 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:05:31.129498 10514 catalog_manager.cc:1347] Generated new cluster ID: bff62c917c74433d9c307146ae505e41
I20260925 22:05:31.129547 10514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:05:31.155681 10514 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:05:31.156443 10514 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:05:31.162458 10514 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42c71141edae4cbd9faf4f38e726ccf2: Generated new TSK 0
I20260925 22:05:31.162664 10514 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:05:31.173338 10507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:31.169716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59652 (local address 127.5.0.126:36045)
0925 22:05:31.169773 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:31.169776 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:31.169869 (+ 93us) server_negotiation.cc:407] Connection header received
0925 22:05:31.169924 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:31.169926 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:31.169937 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:31.170020 (+ 83us) server_negotiation.cc:226] Negotiated authn=JWT
0925 22:05:31.170111 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:31.170909 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:31.171636 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:31.171998 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:31.172070 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:31.172195 (+ 125us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 22:05:31.173128 (+ 933us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39119/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0925 22:05:31.173266 (+ 138us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59652: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39119/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":33}
W20260925 22:05:31.173401 10507 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59652: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39119/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260925 22:05:31.175230 10533 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:31.169621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:36045 (local address 127.0.0.1:59652)
0925 22:05:31.169801 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:31.169811 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:31.169869 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:31.170018 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:31.170019 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:31.170021 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0925 22:05:31.170070 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:31.170072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:31.170981 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:31.170982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:31.171562 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:31.171564 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:31.172066 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:31.172072 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:31.172115 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:31.172119 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 22:05:31.173236 (+ 1117us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39119/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0925 22:05:31.175130 (+ 1894us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.0.126:36045: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39119/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":1702,"thread_start_us":116,"threads_started":1}
I20260925 22:05:31.175392 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 10437
E20260925 22:05:31.184096 10534 webserver.cc:533] Webserver: sslize failed:
E20260925 22:05:31.184159 10534 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-25T22:05:31Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (213 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-25T22:05:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:05:31Z Disabled control of system clock
I20260925 22:05:31.219254 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:44327
--webserver_interface=127.5.0.126
--webserver_port=0
--builtin_ntp_servers=127.5.0.84:41417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:44327
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373908987421-5121-0/testchainca.cert with env {}
W20260925 22:05:31.299191 10538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:31.299415 10538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:31.299444 10538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:31.300702 10538 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:05:31.300742 10538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:31.300757 10538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:31.300771 10538 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:05:31.300786 10538 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 22:05:31.300806 10538 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 22:05:31.300820 10538 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260925 22:05:31.302423 10538 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:41417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:44327
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:44327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373908987421-5121-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:31.302645 10538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:31.302897 10538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:31.307545 10544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:31.307556 10543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:31.307556 10546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:31.307859 10538 server_base.cc:1034] running on GCE node
I20260925 22:05:31.308135 10538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:31.308414 10538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:31.309559 10538 hybrid_clock.cc:648] HybridClock initialized: now 1790373931309548 us; error 28 us; skew 500 ppm
I20260925 22:05:31.310775 10538 webserver.cc:460] Webserver started at http://127.5.0.126:44483/ using document root <none> and password file <none>
I20260925 22:05:31.310997 10538 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:31.311043 10538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:31.311210 10538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:31.312338 10538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "6b9c7ff69dcc4137b055329c76a3965e"
format_stamp: "Formatted at 2026-09-25 22:05:31 on dist-test-slave-7mqr"
server_key: "bf2e62dba7bc5ee725bb811ff94d231c"
server_key_iv: "fbd1e6f7838106dde1568893cb0f8c7e"
server_key_version: "encryptionkey@0"
I20260925 22:05:31.312678 10538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "6b9c7ff69dcc4137b055329c76a3965e"
format_stamp: "Formatted at 2026-09-25 22:05:31 on dist-test-slave-7mqr"
server_key: "bf2e62dba7bc5ee725bb811ff94d231c"
server_key_iv: "fbd1e6f7838106dde1568893cb0f8c7e"
server_key_version: "encryptionkey@0"
I20260925 22:05:31.313971 10538 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:05:31.314661 10552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:31.314831 10538 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:05:31.314921 10538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "6b9c7ff69dcc4137b055329c76a3965e"
format_stamp: "Formatted at 2026-09-25 22:05:31 on dist-test-slave-7mqr"
server_key: "bf2e62dba7bc5ee725bb811ff94d231c"
server_key_iv: "fbd1e6f7838106dde1568893cb0f8c7e"
server_key_version: "encryptionkey@0"
I20260925 22:05:31.314996 10538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:31.328742 10538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:31.329030 10538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:31.329166 10538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:31.332947 10538 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:44327
I20260925 22:05:31.332978 10604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:44327 every 8 connection(s)
I20260925 22:05:31.333339 10538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:05:31.334262 10605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:31.336833 10605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b9c7ff69dcc4137b055329c76a3965e: Bootstrap starting.
I20260925 22:05:31.337494 10605 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b9c7ff69dcc4137b055329c76a3965e: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:31.337795 10605 log.cc:824] T 00000000000000000000000000000000 P 6b9c7ff69dcc4137b055329c76a3965e: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:31.338472 10605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b9c7ff69dcc4137b055329c76a3965e: No bootstrap required, opened a new log
I20260925 22:05:31.340127 10605 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b9c7ff69dcc4137b055329c76a3965e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9c7ff69dcc4137b055329c76a3965e" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 44327 } }
I20260925 22:05:31.340267 10605 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b9c7ff69dcc4137b055329c76a3965e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:31.340318 10605 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b9c7ff69dcc4137b055329c76a3965e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b9c7ff69dcc4137b055329c76a3965e, State: Initialized, Role: FOLLOWER
I20260925 22:05:31.340476 10605 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b9c7ff69dcc4137b055329c76a3965e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9c7ff69dcc4137b055329c76a3965e" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 44327 } }
I20260925 22:05:31.340538 10605 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b9c7ff69dcc4137b055329c76a3965e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:05:31.340574 10605 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b9c7ff69dcc4137b055329c76a3965e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:05:31.340621 10605 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b9c7ff69dcc4137b055329c76a3965e [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:31.341164 10605 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b9c7ff69dcc4137b055329c76a3965e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9c7ff69dcc4137b055329c76a3965e" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 44327 } }
I20260925 22:05:31.341275 10605 leader_election.cc:302] T 00000000000000000000000000000000 P 6b9c7ff69dcc4137b055329c76a3965e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b9c7ff69dcc4137b055329c76a3965e; no voters:
I20260925 22:05:31.341447 10605 leader_election.cc:288] T 00000000000000000000000000000000 P 6b9c7ff69dcc4137b055329c76a3965e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:05:31.341475 10608 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b9c7ff69dcc4137b055329c76a3965e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:31.341622 10608 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b9c7ff69dcc4137b055329c76a3965e [term 1 LEADER]: Becoming Leader. State: Replica: 6b9c7ff69dcc4137b055329c76a3965e, State: Running, Role: LEADER
I20260925 22:05:31.341710 10605 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b9c7ff69dcc4137b055329c76a3965e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:05:31.341741 10608 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b9c7ff69dcc4137b055329c76a3965e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9c7ff69dcc4137b055329c76a3965e" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 44327 } }
I20260925 22:05:31.342139 10609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b9c7ff69dcc4137b055329c76a3965e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b9c7ff69dcc4137b055329c76a3965e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9c7ff69dcc4137b055329c76a3965e" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 44327 } } }
I20260925 22:05:31.342249 10610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b9c7ff69dcc4137b055329c76a3965e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b9c7ff69dcc4137b055329c76a3965e. Latest consensus state: current_term: 1 leader_uuid: "6b9c7ff69dcc4137b055329c76a3965e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9c7ff69dcc4137b055329c76a3965e" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 44327 } } }
I20260925 22:05:31.342339 10610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b9c7ff69dcc4137b055329c76a3965e [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:31.342306 10609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b9c7ff69dcc4137b055329c76a3965e [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:31.342680 10617 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:05:31.343187 10617 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:05:31.343348 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 10538
I20260925 22:05:31.343451 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:05:31.343644 5121 external_mini_cluster.cc:1428] Setting key 950448f18d9674cd0f91ab35d3670936
I20260925 22:05:31.344899 10617 catalog_manager.cc:1347] Generated new cluster ID: cd4fbdc96bc64e889b9f6fdda810a802
I20260925 22:05:31.344969 10617 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:05:31.352883 10627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:31.344122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43652 (local address 127.5.0.126:44327)
0925 22:05:31.344317 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:31.344327 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:31.344359 (+ 32us) server_negotiation.cc:407] Connection header received
0925 22:05:31.344473 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:31.344480 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:31.344548 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:31.344704 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:31.348037 (+ 3333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:31.349412 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:31.350578 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:31.351180 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:31.351287 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:31.352058 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:31.352061 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:31.352062 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:05:31.352272 (+ 210us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:31.352311 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:05:31.352324 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:31.352388 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:31.352473 (+ 85us) server_negotiation.cc:299] Negotiation successful
0925 22:05:31.352640 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":124,"thread_start_us":91,"threads_started":1}
I20260925 22:05:31.353518 10617 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:05:31.353747 5121 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:05:31.354097 10617 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:05:31.361336 10617 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b9c7ff69dcc4137b055329c76a3965e: Generated new TSK 0
I20260925 22:05:31.361523 10617 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:05:31.419442 10627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:31.367787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43668 (local address 127.5.0.126:44327)
0925 22:05:31.367834 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:31.367837 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:31.367877 (+ 40us) server_negotiation.cc:407] Connection header received
0925 22:05:31.367979 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:31.367980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:31.367993 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:31.368073 (+ 80us) server_negotiation.cc:226] Negotiated authn=JWT
0925 22:05:31.368168 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:31.368959 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:31.369717 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:31.370046 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:31.370117 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:31.370260 (+ 143us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 22:05:31.419148 (+ 48888us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0925 22:05:31.419358 (+ 210us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43668: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":24}
W20260925 22:05:31.419524 10627 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43668: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260925 22:05:31.419629 10634 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:31.367681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:44327 (local address 127.0.0.1:43668)
0925 22:05:31.367820 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:31.367831 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:31.367909 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:31.368071 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:31.368072 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:31.368073 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0925 22:05:31.368119 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:31.368121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:31.369048 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:31.369049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:31.369636 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:31.369637 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:31.370114 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:31.370120 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:31.370160 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:31.370164 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 22:05:31.419299 (+ 49135us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0925 22:05:31.419434 (+ 135us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.0.126:44327: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":93,"threads_started":1}
I20260925 22:05:31.420068 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 10538
2026-09-25T22:05:31Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (225 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (86 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (86 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:05:31 dist-test-slave-7mqr krb5kdc[10646](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:05:31 dist-test-slave-7mqr krb5kdc[10646](info): set up 2 sockets
Sep 25 22:05:31 dist-test-slave-7mqr krb5kdc[10646](info): commencing operation
krb5kdc: starting...
W20260925 22:05:32.628171 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:05:32 dist-test-slave-7mqr krb5kdc[10646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:05:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:05:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.0.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
I20260925 22:05:32.746232 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:41883
--webserver_interface=127.5.0.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:46421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:41883
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:32.824764 10659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:32.824949 10659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:32.824970 10659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:32.826078 10659 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:05:32.826113 10659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:32.826128 10659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:32.826139 10659 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:05:32.826151 10659 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:05:32.827518 10659 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:46421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:41883
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:41883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:32.827693 10659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:32.827852 10659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:32.832496 10665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:32.832643 10659 server_base.cc:1034] running on GCE node
W20260925 22:05:32.832623 10664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:32.832726 10667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:32.832993 10659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:32.833259 10659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:32.834410 10659 hybrid_clock.cc:648] HybridClock initialized: now 1790373932834406 us; error 33 us; skew 500 ppm
Sep 25 22:05:32 dist-test-slave-7mqr krb5kdc[10646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373932, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:32.836864 10659 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:05:32.837387 10659 webserver.cc:460] Webserver started at http://127.5.0.126:34497/ using document root <none> and password file <none>
I20260925 22:05:32.837623 10659 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:32.837694 10659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:32.837824 10659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:32.838843 10659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "ed9ed69a8a354b8f9a2c69bdcc8d4f31"
format_stamp: "Formatted at 2026-09-25 22:05:32 on dist-test-slave-7mqr"
server_key: "a366267ec666ddf2be3707fbf4db498a"
server_key_iv: "f6d206175a9627e049faa0a9d594f7f7"
server_key_version: "encryptionkey@0"
I20260925 22:05:32.839195 10659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "ed9ed69a8a354b8f9a2c69bdcc8d4f31"
format_stamp: "Formatted at 2026-09-25 22:05:32 on dist-test-slave-7mqr"
server_key: "a366267ec666ddf2be3707fbf4db498a"
server_key_iv: "f6d206175a9627e049faa0a9d594f7f7"
server_key_version: "encryptionkey@0"
I20260925 22:05:32.840575 10659 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:05:32.841266 10674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:32.841424 10659 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:05:32.841516 10659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "ed9ed69a8a354b8f9a2c69bdcc8d4f31"
format_stamp: "Formatted at 2026-09-25 22:05:32 on dist-test-slave-7mqr"
server_key: "a366267ec666ddf2be3707fbf4db498a"
server_key_iv: "f6d206175a9627e049faa0a9d594f7f7"
server_key_version: "encryptionkey@0"
I20260925 22:05:32.841593 10659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:32.849220 10659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:32.851398 10659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:32.851554 10659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:32.855382 10659 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:41883
I20260925 22:05:32.855413 10726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:41883 every 8 connection(s)
I20260925 22:05:32.855777 10659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:05:32.856658 10727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:32.858855 10727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed9ed69a8a354b8f9a2c69bdcc8d4f31: Bootstrap starting.
I20260925 22:05:32.859395 10727 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed9ed69a8a354b8f9a2c69bdcc8d4f31: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:32.859632 10727 log.cc:824] T 00000000000000000000000000000000 P ed9ed69a8a354b8f9a2c69bdcc8d4f31: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:32.860304 10727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed9ed69a8a354b8f9a2c69bdcc8d4f31: No bootstrap required, opened a new log
I20260925 22:05:32.860796 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 10659
I20260925 22:05:32.860920 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:05:32.861114 5121 external_mini_cluster.cc:1428] Setting key 894c0c54ec4cf7d8941d2dd1def163a0
I20260925 22:05:32.862399 10727 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed9ed69a8a354b8f9a2c69bdcc8d4f31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed9ed69a8a354b8f9a2c69bdcc8d4f31" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41883 } }
I20260925 22:05:32.862567 10727 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed9ed69a8a354b8f9a2c69bdcc8d4f31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:32.862619 10727 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed9ed69a8a354b8f9a2c69bdcc8d4f31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed9ed69a8a354b8f9a2c69bdcc8d4f31, State: Initialized, Role: FOLLOWER
I20260925 22:05:32.862752 10727 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed9ed69a8a354b8f9a2c69bdcc8d4f31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed9ed69a8a354b8f9a2c69bdcc8d4f31" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41883 } }
I20260925 22:05:32.862839 10727 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed9ed69a8a354b8f9a2c69bdcc8d4f31 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:05:32.862872 10727 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed9ed69a8a354b8f9a2c69bdcc8d4f31 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:05:32.862922 10727 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed9ed69a8a354b8f9a2c69bdcc8d4f31 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:32.863965 10727 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed9ed69a8a354b8f9a2c69bdcc8d4f31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed9ed69a8a354b8f9a2c69bdcc8d4f31" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41883 } }
I20260925 22:05:32.864073 10727 leader_election.cc:302] T 00000000000000000000000000000000 P ed9ed69a8a354b8f9a2c69bdcc8d4f31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed9ed69a8a354b8f9a2c69bdcc8d4f31; no voters:
I20260925 22:05:32.864244 10727 leader_election.cc:288] T 00000000000000000000000000000000 P ed9ed69a8a354b8f9a2c69bdcc8d4f31 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:05:32.864279 10732 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed9ed69a8a354b8f9a2c69bdcc8d4f31 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:32.864392 10732 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed9ed69a8a354b8f9a2c69bdcc8d4f31 [term 1 LEADER]: Becoming Leader. State: Replica: ed9ed69a8a354b8f9a2c69bdcc8d4f31, State: Running, Role: LEADER
I20260925 22:05:32.864475 10732 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed9ed69a8a354b8f9a2c69bdcc8d4f31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed9ed69a8a354b8f9a2c69bdcc8d4f31" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41883 } }
I20260925 22:05:32.864516 10727 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed9ed69a8a354b8f9a2c69bdcc8d4f31 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:05:32.864941 10732 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed9ed69a8a354b8f9a2c69bdcc8d4f31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed9ed69a8a354b8f9a2c69bdcc8d4f31. Latest consensus state: current_term: 1 leader_uuid: "ed9ed69a8a354b8f9a2c69bdcc8d4f31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed9ed69a8a354b8f9a2c69bdcc8d4f31" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41883 } } }
I20260925 22:05:32.865022 10732 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed9ed69a8a354b8f9a2c69bdcc8d4f31 [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:32.865185 10733 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed9ed69a8a354b8f9a2c69bdcc8d4f31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed9ed69a8a354b8f9a2c69bdcc8d4f31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed9ed69a8a354b8f9a2c69bdcc8d4f31" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41883 } } }
I20260925 22:05:32.865258 10733 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed9ed69a8a354b8f9a2c69bdcc8d4f31 [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:32.865391 10738 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:05:32.865921 10738 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:05:32.867828 10738 catalog_manager.cc:1347] Generated new cluster ID: 6f7352a35da048a69c32a433fc1ff2b4
I20260925 22:05:32.867900 10738 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:05:32 dist-test-slave-7mqr krb5kdc[10646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:32.872061 10731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:32.861608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44698 (local address 127.5.0.126:41883)
0925 22:05:32.861791 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:32.861800 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:32.861831 (+ 31us) server_negotiation.cc:407] Connection header received
0925 22:05:32.862374 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:32.862380 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:32.862424 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:32.862535 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:32.863356 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:32.864478 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:32.865525 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:32.866150 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:32.866582 (+ 432us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:32.868842 (+ 2260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:32.868846 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:32.868846 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:32.868857 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:32.870511 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:32.870883 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:32.870886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:32.870887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:32.870991 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:32.871209 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:32.871210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:32.871210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:32.871442 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:32.871615 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:32.871691 (+ 76us) server_negotiation.cc:299] Negotiation successful
0925 22:05:32.871832 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":102,"threads_started":1}
I20260925 22:05:32.920315 10738 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:05:32.920939 10738 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:05:32.925094 10738 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed9ed69a8a354b8f9a2c69bdcc8d4f31: Generated new TSK 0
I20260925 22:05:32.925297 10738 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.0.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
I20260925 22:05:32.962350 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.65:0
--local_ip_for_outbound_sockets=127.5.0.65
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:41883
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:46421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:33.050248 10753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:33.050467 10753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:33.050508 10753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:33.052454 10753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:33.052529 10753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:33.052572 10753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.65
I20260925 22:05:33.054914 10753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:46421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:41883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.65
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:33.055168 10753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:33.055456 10753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:33.060627 10759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:33.060648 10758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:33.060652 10761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:33.060770 10753 server_base.cc:1034] running on GCE node
I20260925 22:05:33.061069 10753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:33.061329 10753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:33.062536 10753 hybrid_clock.cc:648] HybridClock initialized: now 1790373933062521 us; error 26 us; skew 500 ppm
Sep 25 22:05:33 dist-test-slave-7mqr krb5kdc[10646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373933, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:33.065217 10753 init.cc:377] Logged in from keytab as kudu/127.5.0.65@KRBTEST.COM (short username kudu)
I20260925 22:05:33.065725 10753 webserver.cc:460] Webserver started at http://127.5.0.65:33295/ using document root <none> and password file <none>
I20260925 22:05:33.065950 10753 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:33.066011 10753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:33.066164 10753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:33.067257 10753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-0/data/instance:
uuid: "392f3fb5ac934371850e19e11a51e264"
format_stamp: "Formatted at 2026-09-25 22:05:33 on dist-test-slave-7mqr"
server_key: "6834a4adfb8500cf27a852222e38f619"
server_key_iv: "08889deefac9867fdbe3d2946d6b2137"
server_key_version: "encryptionkey@0"
I20260925 22:05:33.067603 10753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance:
uuid: "392f3fb5ac934371850e19e11a51e264"
format_stamp: "Formatted at 2026-09-25 22:05:33 on dist-test-slave-7mqr"
server_key: "6834a4adfb8500cf27a852222e38f619"
server_key_iv: "08889deefac9867fdbe3d2946d6b2137"
server_key_version: "encryptionkey@0"
I20260925 22:05:33.068954 10753 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:05:33.069638 10768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:33.069806 10753 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:05:33.069898 10753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-0/wal
uuid: "392f3fb5ac934371850e19e11a51e264"
format_stamp: "Formatted at 2026-09-25 22:05:33 on dist-test-slave-7mqr"
server_key: "6834a4adfb8500cf27a852222e38f619"
server_key_iv: "08889deefac9867fdbe3d2946d6b2137"
server_key_version: "encryptionkey@0"
I20260925 22:05:33.069978 10753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:33.088122 10753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:33.089916 10753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:33.090077 10753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:33.090313 10753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:33.090606 10753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:33.090655 10753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:33.090690 10753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:33.090718 10753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:33.096342 10753 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.65:38057
I20260925 22:05:33.096393 10881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.65:38057 every 8 connection(s)
I20260925 22:05:33.096738 10753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
I20260925 22:05:33.097606 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 10753
I20260925 22:05:33.097729 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance
I20260925 22:05:33.097908 5121 external_mini_cluster.cc:1428] Setting key 421e8e87d1af2ae50d8278080412dc33
Sep 25 22:05:33 dist-test-slave-7mqr krb5kdc[10646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373933, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.0.66@KRBTEST.COM; defaulting to no policy
I20260925 22:05:33.106956 10731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:33.097475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:55177 (local address 127.5.0.126:41883)
0925 22:05:33.097548 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:33.097551 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:33.097719 (+ 168us) server_negotiation.cc:407] Connection header received
0925 22:05:33.098959 (+ 1240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:33.098960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:33.098973 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:33.099013 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:33.099728 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:33.100296 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:33.101305 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:33.101526 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:33.101600 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:33.104192 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:33.104193 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:33.104193 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:33.104200 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:33.105577 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:33.106052 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:33.106053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:33.106053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:33.106101 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:33.106371 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:33.106371 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:33.106372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:33.106486 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:33.106539 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:33.106814 (+ 275us) server_negotiation.cc:299] Negotiation successful
0925 22:05:33.106888 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 22:05:33.107105 10884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:33.097426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41883 (local address 127.5.0.65:55177)
0925 22:05:33.097591 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:33.097632 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:33.098850 (+ 1218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:33.099092 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:33.099096 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:33.099358 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:33.099632 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:33.099637 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:33.100391 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:33.100392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:33.101202 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:33.101204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:33.101610 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:33.101611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:33.101743 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:33.102283 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:33.102293 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:33.104095 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:33.105661 (+ 1566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:33.105668 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:33.105671 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:33.105981 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:33.106161 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:33.106162 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:33.106162 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:33.106307 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:33.106552 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:33.106559 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:33.106703 (+ 144us) client_negotiation.cc:769] Sending connection context
0925 22:05:33.106792 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 22:05:33.106915 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":87,"threads_started":1}
I20260925 22:05:33.107571 10882 heartbeater.cc:347] Connected to a master server at 127.5.0.126:41883
I20260925 22:05:33.107681 10882 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:33.107863 10882 heartbeater.cc:511] Master 127.5.0.126:41883 requested a full tablet report, sending...
I20260925 22:05:33.108521 10691 ts_manager.cc:194] Registered new tserver with Master: 392f3fb5ac934371850e19e11a51e264 (127.5.0.65:38057)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.66@KRBTEST.COM" created.
I20260925 22:05:33.109294 10691 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.65@KRBTEST.COM'} at 127.5.0.65:55177
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
I20260925 22:05:33.122869 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.66:0
--local_ip_for_outbound_sockets=127.5.0.66
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:41883
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:46421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:33.202975 10887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:33.203181 10887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:33.203212 10887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:33.204329 10887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:33.204363 10887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:33.204389 10887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.66
I20260925 22:05:33.205778 10887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:46421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:41883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.66
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:33.205960 10887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:33.206126 10887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:33.211119 10892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:33.211117 10895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:33.211302 10887 server_base.cc:1034] running on GCE node
W20260925 22:05:33.211164 10893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:33.211580 10887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:33.211791 10887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:33.212924 10887 hybrid_clock.cc:648] HybridClock initialized: now 1790373933212906 us; error 32 us; skew 500 ppm
Sep 25 22:05:33 dist-test-slave-7mqr krb5kdc[10646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373933, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:33.215387 10887 init.cc:377] Logged in from keytab as kudu/127.5.0.66@KRBTEST.COM (short username kudu)
I20260925 22:05:33.215802 10887 webserver.cc:460] Webserver started at http://127.5.0.66:35425/ using document root <none> and password file <none>
I20260925 22:05:33.216032 10887 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:33.216075 10887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:33.216173 10887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:33.217226 10887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-1/data/instance:
uuid: "01ba02a638ee41188136b0afaa6eab98"
format_stamp: "Formatted at 2026-09-25 22:05:33 on dist-test-slave-7mqr"
server_key: "01ea702063e0576e853e0e53d3eda84a"
server_key_iv: "34aa04a759c647146acda0cda5ecb4d5"
server_key_version: "encryptionkey@0"
I20260925 22:05:33.217523 10887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance:
uuid: "01ba02a638ee41188136b0afaa6eab98"
format_stamp: "Formatted at 2026-09-25 22:05:33 on dist-test-slave-7mqr"
server_key: "01ea702063e0576e853e0e53d3eda84a"
server_key_iv: "34aa04a759c647146acda0cda5ecb4d5"
server_key_version: "encryptionkey@0"
I20260925 22:05:33.218737 10887 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:05:33.219380 10902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:33.219535 10887 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:05:33.219632 10887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-1/wal
uuid: "01ba02a638ee41188136b0afaa6eab98"
format_stamp: "Formatted at 2026-09-25 22:05:33 on dist-test-slave-7mqr"
server_key: "01ea702063e0576e853e0e53d3eda84a"
server_key_iv: "34aa04a759c647146acda0cda5ecb4d5"
server_key_version: "encryptionkey@0"
I20260925 22:05:33.219700 10887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:33.228693 10887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:33.230876 10887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:33.231046 10887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:33.231263 10887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:33.231563 10887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:33.231596 10887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:33.231637 10887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:33.231662 10887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:33.237149 10887 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.66:45667
I20260925 22:05:33.237212 11015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.66:45667 every 8 connection(s)
I20260925 22:05:33.237535 10887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
Sep 25 22:05:33 dist-test-slave-7mqr krb5kdc[10646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373933, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:33.246584 10731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:33.238244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:36089 (local address 127.5.0.126:41883)
0925 22:05:33.238288 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:33.238290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:33.238499 (+ 209us) server_negotiation.cc:407] Connection header received
0925 22:05:33.239469 (+ 970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:33.239471 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:33.239483 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:33.239524 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:33.240206 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:33.240707 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:33.241684 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:33.241880 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:33.241960 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:33.243962 (+ 2002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:33.243963 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:33.243964 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:33.243971 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:33.245180 (+ 1209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:33.245618 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:33.245619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:33.245619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:33.245682 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:33.245935 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:33.245936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:33.245936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:33.246050 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:33.246135 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:33.246396 (+ 261us) server_negotiation.cc:299] Negotiation successful
0925 22:05:33.246494 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:05:33.246780 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:33.238228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41883 (local address 127.5.0.66:36089)
0925 22:05:33.238389 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:33.238422 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:33.239354 (+ 932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:33.239603 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:33.239607 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:33.239869 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:33.240112 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:33.240116 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:33.240796 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:33.240797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:33.241583 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:33.241585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:33.241968 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:33.241969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:33.242091 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:33.242512 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:33.242520 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:33.243847 (+ 1327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:33.245272 (+ 1425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:33.245278 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:33.245280 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:33.245548 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:33.245756 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:33.245757 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:33.245757 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:33.245863 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:33.246125 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:33.246131 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:33.246269 (+ 138us) client_negotiation.cc:769] Sending connection context
0925 22:05:33.246374 (+ 105us) client_negotiation.cc:240] Negotiation successful
0925 22:05:33.246539 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":81,"threads_started":1}
I20260925 22:05:33.247248 11016 heartbeater.cc:347] Connected to a master server at 127.5.0.126:41883
I20260925 22:05:33.247349 11016 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:33.247370 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 10887
I20260925 22:05:33.247463 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance
I20260925 22:05:33.247543 11016 heartbeater.cc:511] Master 127.5.0.126:41883 requested a full tablet report, sending...
I20260925 22:05:33.247668 5121 external_mini_cluster.cc:1428] Setting key 2bc05a0a49ca7d44af142479f9c78260
I20260925 22:05:33.247921 10691 ts_manager.cc:194] Registered new tserver with Master: 01ba02a638ee41188136b0afaa6eab98 (127.5.0.66:45667)
I20260925 22:05:33.248534 10691 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.66@KRBTEST.COM'} at 127.5.0.66:36089
WARNING: no policy specified for kudu/127.5.0.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
I20260925 22:05:33.270680 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.67:0
--local_ip_for_outbound_sockets=127.5.0.67
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:41883
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:46421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:33.351192 11021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:33.351385 11021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:33.351408 11021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:33.352512 11021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:33.352545 11021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:33.352576 11021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.67
I20260925 22:05:33.353977 11021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:46421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:41883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.67
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:33.354161 11021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:33.354337 11021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:33.359093 11029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:33.359113 11026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:33.359211 11027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:33.359319 11021 server_base.cc:1034] running on GCE node
I20260925 22:05:33.359512 11021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:33.359712 11021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:33.360844 11021 hybrid_clock.cc:648] HybridClock initialized: now 1790373933360819 us; error 41 us; skew 500 ppm
Sep 25 22:05:33 dist-test-slave-7mqr krb5kdc[10646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373933, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:33.363163 11021 init.cc:377] Logged in from keytab as kudu/127.5.0.67@KRBTEST.COM (short username kudu)
I20260925 22:05:33.363616 11021 webserver.cc:460] Webserver started at http://127.5.0.67:46773/ using document root <none> and password file <none>
I20260925 22:05:33.363847 11021 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:33.363945 11021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:33.364092 11021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:33.365231 11021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-2/data/instance:
uuid: "815d9f141e0e4a8cab454d0646827d16"
format_stamp: "Formatted at 2026-09-25 22:05:33 on dist-test-slave-7mqr"
server_key: "4e88c0f12bb11313c898c24364738436"
server_key_iv: "59d0e0825ec97ea6796f555c1d06f12b"
server_key_version: "encryptionkey@0"
I20260925 22:05:33.365600 11021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance:
uuid: "815d9f141e0e4a8cab454d0646827d16"
format_stamp: "Formatted at 2026-09-25 22:05:33 on dist-test-slave-7mqr"
server_key: "4e88c0f12bb11313c898c24364738436"
server_key_iv: "59d0e0825ec97ea6796f555c1d06f12b"
server_key_version: "encryptionkey@0"
I20260925 22:05:33.366940 11021 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:05:33.367600 11036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:33.367767 11021 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:05:33.367858 11021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-2/wal
uuid: "815d9f141e0e4a8cab454d0646827d16"
format_stamp: "Formatted at 2026-09-25 22:05:33 on dist-test-slave-7mqr"
server_key: "4e88c0f12bb11313c898c24364738436"
server_key_iv: "59d0e0825ec97ea6796f555c1d06f12b"
server_key_version: "encryptionkey@0"
I20260925 22:05:33.367957 11021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:33.392709 11021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:33.394582 11021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:33.394747 11021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:33.394980 11021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:33.395326 11021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:33.395385 11021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:33.395424 11021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:33.395452 11021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:33.401106 11021 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.67:46553
I20260925 22:05:33.401155 11149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.67:46553 every 8 connection(s)
I20260925 22:05:33.401465 11021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
I20260925 22:05:33.405509 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 11021
I20260925 22:05:33.405612 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance
I20260925 22:05:33.405776 5121 external_mini_cluster.cc:1428] Setting key 64a2eadb019b3939e2b2e8694e59ae1c
Sep 25 22:05:33 dist-test-slave-7mqr krb5kdc[10646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373933, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:33.410825 10731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:33.402171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:55365 (local address 127.5.0.126:41883)
0925 22:05:33.402227 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:33.402229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:33.402502 (+ 273us) server_negotiation.cc:407] Connection header received
0925 22:05:33.403472 (+ 970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:33.403473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:33.403487 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:33.403550 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:33.404241 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:33.404752 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:33.405800 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:33.406085 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:33.406151 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:33.408290 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:33.408292 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:33.408292 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:33.408300 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:33.409477 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:33.409880 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:33.409882 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:33.409882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:33.409944 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:33.410178 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:33.410179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:33.410179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:33.410302 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:33.410376 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:33.410643 (+ 267us) server_negotiation.cc:299] Negotiation successful
0925 22:05:33.410723 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 22:05:33.411029 11152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:33.402149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41883 (local address 127.5.0.67:55365)
0925 22:05:33.402347 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:33.402413 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:33.403355 (+ 942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:33.403642 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:33.403646 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:33.403942 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:33.404131 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:33.404136 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:33.404847 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:33.404848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:33.405693 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:33.405695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:33.406171 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:33.406172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:33.406296 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:33.406844 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:33.406852 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:33.408193 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:33.409573 (+ 1380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:33.409579 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:33.409581 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:33.409807 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:33.410008 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:33.410009 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:33.410010 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:33.410113 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:33.410367 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:33.410373 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:33.410513 (+ 140us) client_negotiation.cc:769] Sending connection context
0925 22:05:33.410620 (+ 107us) client_negotiation.cc:240] Negotiation successful
0925 22:05:33.410791 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":95,"threads_started":1}
I20260925 22:05:33.411455 11150 heartbeater.cc:347] Connected to a master server at 127.5.0.126:41883
I20260925 22:05:33.411547 11150 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:33.411687 11150 heartbeater.cc:511] Master 127.5.0.126:41883 requested a full tablet report, sending...
I20260925 22:05:33.412037 10691 ts_manager.cc:194] Registered new tserver with Master: 815d9f141e0e4a8cab454d0646827d16 (127.5.0.67:46553)
I20260925 22:05:33.412421 10691 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.67@KRBTEST.COM'} at 127.5.0.67:55365
I20260925 22:05:33.417383 5121 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/test-admin.keytab.
Sep 25 22:05:33 dist-test-slave-7mqr krb5kdc[10646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:33.432168 5121 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260925 22:05:33.433745 11157 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:33.432989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58356 (local address 127.5.0.65:38057)
0925 22:05:33.433174 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:33.433179 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:33.433189 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:05:33.433223 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:33.433225 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:33.433239 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:33.433304 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:33.433636 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58356: BlockingRecv error: recv got EOF from 127.0.0.1:58356 (error 108)
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":111,"threads_started":1}
W20260925 22:05:33.433844 5121 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/krb5cc' of type FILE
Sep 25 22:05:33 dist-test-slave-7mqr krb5kdc[10646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:33.434829 5121 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 22:05:33 dist-test-slave-7mqr krb5kdc[10646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:33.435676 5121 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 22:05:33 dist-test-slave-7mqr krb5kdc[10646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.65@KRBTEST.COM
I20260925 22:05:33.443377 11157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:33.436245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58368 (local address 127.5.0.65:38057)
0925 22:05:33.436298 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:33.436301 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:33.436358 (+ 57us) server_negotiation.cc:407] Connection header received
0925 22:05:33.436440 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:33.436441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:33.436453 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:33.436513 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:33.436915 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:33.437636 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:33.438405 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:33.438723 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:33.438800 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:33.440761 (+ 1961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:33.440768 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:33.440768 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:33.440776 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:33.442218 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:33.442573 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:33.442576 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:33.442576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:33.442634 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:33.442834 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:33.442835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:33.442835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:33.442965 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:33.443109 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:33.443181 (+ 72us) server_negotiation.cc:299] Negotiation successful
0925 22:05:33.443267 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260925 22:05:33.444985 11157 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:33.444490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58380 (local address 127.5.0.65:38057)
0925 22:05:33.444524 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:33.444526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:33.444557 (+ 31us) server_negotiation.cc:407] Connection header received
0925 22:05:33.444601 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:33.444602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:33.444609 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:33.444656 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:33.444937 (+ 281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58380: BlockingRecv error: recv got EOF from 127.0.0.1:58380 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260925 22:05:33.445144 5121 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/krb5cc)
Sep 25 22:05:33 dist-test-slave-7mqr krb5kdc[10646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:33.446134 5121 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 22:05:33 dist-test-slave-7mqr krb5kdc[10646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:33.447016 5121 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 22:05:33 dist-test-slave-7mqr krb5kdc[10646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.65@KRBTEST.COM
I20260925 22:05:33.453950 11157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:33.447559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58388 (local address 127.5.0.65:38057)
0925 22:05:33.447608 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:33.447610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:33.447619 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:05:33.447681 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:33.447682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:33.447691 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:33.447747 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:33.448151 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:33.448609 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:33.449281 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:33.449489 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:33.449561 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:33.451541 (+ 1980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:33.451542 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:33.451542 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:33.451549 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:33.452873 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:33.453202 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:33.453204 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:33.453204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:33.453258 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:33.453454 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:33.453454 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:33.453455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:33.453547 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:33.453636 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:33.453776 (+ 140us) server_negotiation.cc:299] Negotiation successful
0925 22:05:33.453852 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
W20260925 22:05:33.455287 11157 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:33.454753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58390 (local address 127.5.0.65:38057)
0925 22:05:33.454785 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:33.454787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:33.454848 (+ 61us) server_negotiation.cc:407] Connection header received
0925 22:05:33.454905 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:33.454906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:33.454913 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:33.454959 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:33.455226 (+ 267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58390: BlockingRecv error: recv got EOF from 127.0.0.1:58390 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260925 22:05:33.455391 5121 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373908987421-5121-0/krb5kdc/krb5cc' of type FILE
Sep 25 22:05:33 dist-test-slave-7mqr krb5kdc[10646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:33.456372 5121 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 22:05:33 dist-test-slave-7mqr krb5kdc[10646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:33.457211 5121 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 22:05:33 dist-test-slave-7mqr krb5kdc[10646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.65@KRBTEST.COM
I20260925 22:05:33.463107 11157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:33.457662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58398 (local address 127.5.0.65:38057)
0925 22:05:33.457711 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:33.457712 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:33.457742 (+ 30us) server_negotiation.cc:407] Connection header received
0925 22:05:33.457795 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:33.457796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:33.457803 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:33.457850 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:33.458149 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:33.458592 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:33.459208 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:33.459376 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:33.459423 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:33.461248 (+ 1825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:33.461249 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:33.461249 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:33.461254 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:33.462333 (+ 1079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:33.462608 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:33.462609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:33.462609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:33.462653 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:33.462795 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:33.462795 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:33.462796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:33.462867 (+ 71us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:33.462924 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:33.463005 (+ 81us) server_negotiation.cc:299] Negotiation successful
0925 22:05:33.463052 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 22:05:33.463774 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 10753
I20260925 22:05:33.468632 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 10887
I20260925 22:05:33.473014 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 11021
I20260925 22:05:33.477133 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 10659
2026-09-25T22:05:33Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1885 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:05:33 dist-test-slave-7mqr krb5kdc[11170](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:05:33 dist-test-slave-7mqr krb5kdc[11170](info): set up 2 sockets
Sep 25 22:05:33 dist-test-slave-7mqr krb5kdc[11170](info): commencing operation
krb5kdc: starting...
W20260925 22:05:34.512132 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:05:34 dist-test-slave-7mqr krb5kdc[11170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373934, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:05:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:05:34Z Disabled control of system clock
WARNING: no policy specified for oryx/127.5.0.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.0.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.0.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.126.keytab.
Entry for principal oryx/127.5.0.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.126.keytab.
I20260925 22:05:34.627279 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:41553
--webserver_interface=127.5.0.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.126.keytab
--principal=oryx/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:39513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:41553
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:34.706691 11183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:34.706899 11183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:34.706923 11183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:34.708106 11183 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:05:34.708148 11183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:34.708163 11183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:34.708213 11183 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:05:34.708238 11183 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:05:34.709705 11183 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:39513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:41553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.126.keytab
--principal=oryx/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:41553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11183
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:34.709954 11183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:34.710204 11183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:34.714932 11191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:34.714922 11189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:34.714921 11188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:34.714948 11183 server_base.cc:1034] running on GCE node
I20260925 22:05:34.715358 11183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:34.715677 11183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:34.716845 11183 hybrid_clock.cc:648] HybridClock initialized: now 1790373934716813 us; error 45 us; skew 500 ppm
Sep 25 22:05:34 dist-test-slave-7mqr krb5kdc[11170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373934, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:34.719324 11183 init.cc:377] Logged in from keytab as oryx/127.5.0.126@KRBTEST.COM (short username oryx)
I20260925 22:05:34.719779 11183 webserver.cc:460] Webserver started at http://127.5.0.126:45391/ using document root <none> and password file <none>
I20260925 22:05:34.720067 11183 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:34.720135 11183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:34.720273 11183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:34.721407 11183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "8faad6f3402b4d1794676034a93d324a"
format_stamp: "Formatted at 2026-09-25 22:05:34 on dist-test-slave-7mqr"
server_key: "67258fdf7ba6c6b99e9800f880f2c33d"
server_key_iv: "64d7a996312d74d16ee3f39083656ff3"
server_key_version: "encryptionkey@0"
I20260925 22:05:34.721761 11183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "8faad6f3402b4d1794676034a93d324a"
format_stamp: "Formatted at 2026-09-25 22:05:34 on dist-test-slave-7mqr"
server_key: "67258fdf7ba6c6b99e9800f880f2c33d"
server_key_iv: "64d7a996312d74d16ee3f39083656ff3"
server_key_version: "encryptionkey@0"
I20260925 22:05:34.723043 11183 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:05:34.723711 11198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:34.723925 11183 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:05:34.724067 11183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "8faad6f3402b4d1794676034a93d324a"
format_stamp: "Formatted at 2026-09-25 22:05:34 on dist-test-slave-7mqr"
server_key: "67258fdf7ba6c6b99e9800f880f2c33d"
server_key_iv: "64d7a996312d74d16ee3f39083656ff3"
server_key_version: "encryptionkey@0"
I20260925 22:05:34.724148 11183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:34.751256 11183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:34.753239 11183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:34.753405 11183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:34.757160 11183 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:41553
I20260925 22:05:34.757225 11250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:41553 every 8 connection(s)
I20260925 22:05:34.757567 11183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:05:34.758502 11251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:34.760779 11251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a: Bootstrap starting.
I20260925 22:05:34.761356 11251 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:34.761636 11251 log.cc:824] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:34.761931 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 11183
I20260925 22:05:34.762054 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:05:34.762252 5121 external_mini_cluster.cc:1428] Setting key 4d0fa5f5518cec93b4b22ad2aad8e917
I20260925 22:05:34.762372 11251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a: No bootstrap required, opened a new log
I20260925 22:05:34.764230 11251 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } }
I20260925 22:05:34.764386 11251 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:34.764425 11251 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8faad6f3402b4d1794676034a93d324a, State: Initialized, Role: FOLLOWER
I20260925 22:05:34.764581 11251 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } }
I20260925 22:05:34.764673 11251 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:05:34.764710 11251 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:05:34.764760 11251 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:34.765740 11251 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } }
I20260925 22:05:34.765826 11251 leader_election.cc:302] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8faad6f3402b4d1794676034a93d324a; no voters:
I20260925 22:05:34.766010 11251 leader_election.cc:288] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:05:34.766031 11256 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:34.766134 11256 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [term 1 LEADER]: Becoming Leader. State: Replica: 8faad6f3402b4d1794676034a93d324a, State: Running, Role: LEADER
I20260925 22:05:34.766214 11256 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } }
I20260925 22:05:34.766274 11251 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:05:34.766541 11256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8faad6f3402b4d1794676034a93d324a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } }
I20260925 22:05:34.766606 11256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:34.766736 11256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8faad6f3402b4d1794676034a93d324a. Latest consensus state: current_term: 1 leader_uuid: "8faad6f3402b4d1794676034a93d324a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } }
I20260925 22:05:34.766795 11256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:34.767473 11260 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:05:34.768095 11260 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:05:34.769733 11260 catalog_manager.cc:1347] Generated new cluster ID: 812165beba8f4c9c8ac57ab50e1ed30e
I20260925 22:05:34.769789 11260 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:05:34 dist-test-slave-7mqr krb5kdc[11170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373934, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM
I20260925 22:05:34.774065 11255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:34.762753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41748 (local address 127.5.0.126:41553)
0925 22:05:34.762921 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:34.762930 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:34.762972 (+ 42us) server_negotiation.cc:407] Connection header received
0925 22:05:34.764209 (+ 1237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:34.764217 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:34.764270 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:34.764415 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:34.765185 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:34.766315 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:34.767348 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:34.767994 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:34.768098 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:34.770793 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:34.770797 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:34.770797 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:34.770808 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:34.772545 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:34.772917 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:34.772920 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:34.772920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:34.773043 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:34.773243 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:34.773244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:34.773245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:34.773483 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:34.773651 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:34.773717 (+ 66us) server_negotiation.cc:299] Negotiation successful
0925 22:05:34.773853 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":93,"threads_started":1}
I20260925 22:05:34.779510 11260 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:05:34.780104 11260 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:05:34.785698 11260 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a: Generated new TSK 0
I20260925 22:05:34.785885 11260 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.5.0.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.0.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.0.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.65.keytab.
Entry for principal oryx/127.5.0.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.65.keytab.
W20260925 22:05:35.894680 5121 mini_kdc.cc:262] Time spent creating service keytab for oryx/127.5.0.65: real 1.107s user 0.001s sys 0.002s
I20260925 22:05:35.895028 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.65:0
--local_ip_for_outbound_sockets=127.5.0.65
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:41553
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.65.keytab
--principal=oryx/127.5.0.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:39513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:35.979501 11277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:35.979710 11277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:35.979733 11277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:35.980876 11277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:35.980919 11277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:35.980945 11277 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.65
I20260925 22:05:35.982336 11277 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:39513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.65.keytab
--principal=oryx/127.5.0.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:41553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.65
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:35.982522 11277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:35.982693 11277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:35.987458 11282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:35.987474 11285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:35.987529 11283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:35.987877 11277 server_base.cc:1034] running on GCE node
I20260925 22:05:35.988092 11277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:35.988324 11277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:35.989473 11277 hybrid_clock.cc:648] HybridClock initialized: now 1790373935989464 us; error 27 us; skew 500 ppm
Sep 25 22:05:35 dist-test-slave-7mqr krb5kdc[11170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373935, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:35.998330 11277 init.cc:377] Logged in from keytab as oryx/127.5.0.65@KRBTEST.COM (short username oryx)
I20260925 22:05:35.998811 11277 webserver.cc:460] Webserver started at http://127.5.0.65:36943/ using document root <none> and password file <none>
I20260925 22:05:35.999046 11277 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:35.999112 11277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:35.999260 11277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:36.000352 11277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-0/data/instance:
uuid: "2d23d1eb9a374b9a87383727acd62f84"
format_stamp: "Formatted at 2026-09-25 22:05:35 on dist-test-slave-7mqr"
server_key: "2cb7446d9026e315dcbd725ef5828945"
server_key_iv: "3e78d4696cadbfb72c4ce3db99833482"
server_key_version: "encryptionkey@0"
I20260925 22:05:36.000699 11277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance:
uuid: "2d23d1eb9a374b9a87383727acd62f84"
format_stamp: "Formatted at 2026-09-25 22:05:35 on dist-test-slave-7mqr"
server_key: "2cb7446d9026e315dcbd725ef5828945"
server_key_iv: "3e78d4696cadbfb72c4ce3db99833482"
server_key_version: "encryptionkey@0"
I20260925 22:05:36.001981 11277 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 22:05:36.002681 11292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:36.002856 11277 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:05:36.002952 11277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-0/wal
uuid: "2d23d1eb9a374b9a87383727acd62f84"
format_stamp: "Formatted at 2026-09-25 22:05:35 on dist-test-slave-7mqr"
server_key: "2cb7446d9026e315dcbd725ef5828945"
server_key_iv: "3e78d4696cadbfb72c4ce3db99833482"
server_key_version: "encryptionkey@0"
I20260925 22:05:36.003021 11277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:36.011341 11277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:36.013078 11277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:36.013248 11277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:36.013453 11277 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:36.013731 11277 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:36.013782 11277 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:36.013821 11277 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:36.013849 11277 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:36.019411 11277 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.65:44879
I20260925 22:05:36.019452 11405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.65:44879 every 8 connection(s)
I20260925 22:05:36.019766 11277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
Sep 25 22:05:36 dist-test-slave-7mqr krb5kdc[11170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373935, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM
I20260925 22:05:36.028669 11409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:36.020447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:48897 (local address 127.5.0.126:41553)
0925 22:05:36.020574 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:36.020576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:36.020724 (+ 148us) server_negotiation.cc:407] Connection header received
0925 22:05:36.021187 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:36.021187 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:36.021199 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:36.021266 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:36.022204 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:36.022695 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:36.023629 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:36.023835 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:36.023918 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:36.025890 (+ 1972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:36.025891 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:36.025892 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:36.025899 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:36.027479 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:36.027842 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:36.027843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:36.027843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:36.027933 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:36.028146 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:36.028146 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:36.028147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:36.028259 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:36.028313 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:36.028529 (+ 216us) server_negotiation.cc:299] Negotiation successful
0925 22:05:36.028596 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":82,"threads_started":1}
I20260925 22:05:36.028861 11408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:36.020418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.5.0.65:48897)
0925 22:05:36.020580 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:36.020619 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:36.021106 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:36.021378 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:36.021382 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:36.021943 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:36.022125 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:36.022129 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:36.022774 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:36.022775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:36.023557 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:36.023558 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:36.023934 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:36.023935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:36.024051 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:36.024489 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:36.024497 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:36.025794 (+ 1297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:36.027556 (+ 1762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:36.027561 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:36.027562 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:36.027785 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:36.027996 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:36.027997 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:36.027997 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:36.028093 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:36.028312 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:36.028316 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:36.028427 (+ 111us) client_negotiation.cc:769] Sending connection context
0925 22:05:36.028506 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 22:05:36.028624 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":89,"threads_started":1}
I20260925 22:05:36.029397 11406 heartbeater.cc:347] Connected to a master server at 127.5.0.126:41553
I20260925 22:05:36.029506 11406 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:36.029719 11406 heartbeater.cc:511] Master 127.5.0.126:41553 requested a full tablet report, sending...
I20260925 22:05:36.029888 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 11277
I20260925 22:05:36.029994 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance
I20260925 22:05:36.030220 5121 external_mini_cluster.cc:1428] Setting key 069d6e47ba0cc93ff6975874dfa8a36f
I20260925 22:05:36.030330 11215 ts_manager.cc:194] Registered new tserver with Master: 2d23d1eb9a374b9a87383727acd62f84 (127.5.0.65:44879)
I20260925 22:05:36.031340 11215 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.0.65@KRBTEST.COM'} at 127.5.0.65:48897
WARNING: no policy specified for oryx/127.5.0.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.0.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.0.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.66.keytab.
Entry for principal oryx/127.5.0.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.66.keytab.
W20260925 22:05:36.783342 5121 mini_kdc.cc:262] Time spent creating service keytab for oryx/127.5.0.66: real 0.752s user 0.000s sys 0.003s
I20260925 22:05:36.783712 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.66:0
--local_ip_for_outbound_sockets=127.5.0.66
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:41553
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.66.keytab
--principal=oryx/127.5.0.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:39513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:36.865733 11412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:36.865959 11412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:36.865991 11412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:36.867211 11412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:36.867268 11412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:36.867311 11412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.66
I20260925 22:05:36.868882 11412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:39513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.66.keytab
--principal=oryx/127.5.0.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:41553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.66
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:36.869123 11412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:36.869346 11412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:36.874063 11417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:36.874089 11418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:36.874310 11420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:36.874325 11412 server_base.cc:1034] running on GCE node
I20260925 22:05:36.874567 11412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:36.874848 11412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:36.876003 11412 hybrid_clock.cc:648] HybridClock initialized: now 1790373936875985 us; error 31 us; skew 500 ppm
Sep 25 22:05:36 dist-test-slave-7mqr krb5kdc[11170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373936, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:36.878506 11412 init.cc:377] Logged in from keytab as oryx/127.5.0.66@KRBTEST.COM (short username oryx)
I20260925 22:05:36.878945 11412 webserver.cc:460] Webserver started at http://127.5.0.66:36691/ using document root <none> and password file <none>
I20260925 22:05:36.879142 11412 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:36.879181 11412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:36.879285 11412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:36.880323 11412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-1/data/instance:
uuid: "e74b9533a2084cb5a37834fc583b7beb"
format_stamp: "Formatted at 2026-09-25 22:05:36 on dist-test-slave-7mqr"
server_key: "e83eaa054ee9d5dfc73bea8acf6cd53f"
server_key_iv: "7e57ef5e2f986417aa3ef2d533d9aca3"
server_key_version: "encryptionkey@0"
I20260925 22:05:36.880620 11412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance:
uuid: "e74b9533a2084cb5a37834fc583b7beb"
format_stamp: "Formatted at 2026-09-25 22:05:36 on dist-test-slave-7mqr"
server_key: "e83eaa054ee9d5dfc73bea8acf6cd53f"
server_key_iv: "7e57ef5e2f986417aa3ef2d533d9aca3"
server_key_version: "encryptionkey@0"
I20260925 22:05:36.881856 11412 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:05:36.882556 11427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:36.882791 11412 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:05:36.882885 11412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-1/wal
uuid: "e74b9533a2084cb5a37834fc583b7beb"
format_stamp: "Formatted at 2026-09-25 22:05:36 on dist-test-slave-7mqr"
server_key: "e83eaa054ee9d5dfc73bea8acf6cd53f"
server_key_iv: "7e57ef5e2f986417aa3ef2d533d9aca3"
server_key_version: "encryptionkey@0"
I20260925 22:05:36.882941 11412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:36.893119 11412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:36.894925 11412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:36.895035 11412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:36.895202 11412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:36.895443 11412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:36.895473 11412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:36.895494 11412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:36.895506 11412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:36.901033 11412 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.66:34921
I20260925 22:05:36.901117 11540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.66:34921 every 8 connection(s)
I20260925 22:05:36.901404 11412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
Sep 25 22:05:36 dist-test-slave-7mqr krb5kdc[11170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373936, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM
I20260925 22:05:36.908488 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 11412
I20260925 22:05:36.908612 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance
I20260925 22:05:36.908851 5121 external_mini_cluster.cc:1428] Setting key c214802f64c3fff5ed11c0a0e546ff15
I20260925 22:05:36.910841 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:36.902084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:51041 (local address 127.5.0.126:41553)
0925 22:05:36.902216 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:36.902218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:36.902367 (+ 149us) server_negotiation.cc:407] Connection header received
0925 22:05:36.902885 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:36.902886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:36.902899 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:36.902972 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:36.903989 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:36.904504 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:36.905511 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:36.905724 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:36.905783 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:36.907993 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:36.907994 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:36.907995 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:36.908001 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:36.909513 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:36.909883 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:36.909884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:36.909885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:36.909948 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:36.910164 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:36.910165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:36.910165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:36.910308 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:36.910377 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:36.910599 (+ 222us) server_negotiation.cc:299] Negotiation successful
0925 22:05:36.910700 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":63,"threads_started":1}
I20260925 22:05:36.910864 11544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:36.902060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.5.0.66:51041)
0925 22:05:36.902236 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:36.902278 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:36.902781 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:36.903042 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:36.903046 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:36.903627 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:36.903826 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:36.903831 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:36.904599 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:36.904600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:36.905410 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:36.905432 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:36.905795 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:36.905796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:36.905892 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:36.906390 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:36.906398 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:36.907858 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:36.909597 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:36.909603 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:36.909604 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:36.909815 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:36.910014 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:36.910014 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:36.910014 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:36.910108 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:36.910368 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:36.910372 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:36.910481 (+ 109us) client_negotiation.cc:769] Sending connection context
0925 22:05:36.910560 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 22:05:36.910674 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":78,"threads_started":1}
I20260925 22:05:36.911437 11541 heartbeater.cc:347] Connected to a master server at 127.5.0.126:41553
I20260925 22:05:36.911537 11541 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:36.911725 11541 heartbeater.cc:511] Master 127.5.0.126:41553 requested a full tablet report, sending...
I20260925 22:05:36.912128 11215 ts_manager.cc:194] Registered new tserver with Master: e74b9533a2084cb5a37834fc583b7beb (127.5.0.66:34921)
I20260925 22:05:36.912484 11215 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.0.66@KRBTEST.COM'} at 127.5.0.66:51041
WARNING: no policy specified for oryx/127.5.0.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.0.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.0.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.67.keytab.
Entry for principal oryx/127.5.0.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.67.keytab.
I20260925 22:05:36.935216 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.67:0
--local_ip_for_outbound_sockets=127.5.0.67
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:41553
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.67.keytab
--principal=oryx/127.5.0.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:39513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:37.016116 11547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:37.016312 11547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:37.016335 11547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:37.017401 11547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:37.017439 11547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:37.017464 11547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.67
I20260925 22:05:37.018810 11547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:39513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.67.keytab
--principal=oryx/127.5.0.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:41553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.67
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:37.018978 11547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:37.019147 11547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:37.024585 11552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:37.024636 11553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:37.024597 11555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:37.024873 11547 server_base.cc:1034] running on GCE node
I20260925 22:05:37.025051 11547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:37.025277 11547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:37.026437 11547 hybrid_clock.cc:648] HybridClock initialized: now 1790373937026429 us; error 36 us; skew 500 ppm
Sep 25 22:05:37 dist-test-slave-7mqr krb5kdc[11170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373937, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:37.028957 11547 init.cc:377] Logged in from keytab as oryx/127.5.0.67@KRBTEST.COM (short username oryx)
I20260925 22:05:37.029404 11547 webserver.cc:460] Webserver started at http://127.5.0.67:37713/ using document root <none> and password file <none>
I20260925 22:05:37.029632 11547 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:37.029696 11547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:37.029837 11547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:37.030836 11547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-2/data/instance:
uuid: "35c4eb82e71d482aac5e40b60a85dcbc"
format_stamp: "Formatted at 2026-09-25 22:05:37 on dist-test-slave-7mqr"
server_key: "9aec6119736354ca07a5efdb12375443"
server_key_iv: "17e99f26e40e71496c2d42be0d8184f1"
server_key_version: "encryptionkey@0"
I20260925 22:05:37.031179 11547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance:
uuid: "35c4eb82e71d482aac5e40b60a85dcbc"
format_stamp: "Formatted at 2026-09-25 22:05:37 on dist-test-slave-7mqr"
server_key: "9aec6119736354ca07a5efdb12375443"
server_key_iv: "17e99f26e40e71496c2d42be0d8184f1"
server_key_version: "encryptionkey@0"
I20260925 22:05:37.032298 11406 heartbeater.cc:503] Master 127.5.0.126:41553 was elected leader, sending a full tablet report...
I20260925 22:05:37.032624 11547 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:05:37.033368 11562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:37.033536 11547 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:05:37.033625 11547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-2/wal
uuid: "35c4eb82e71d482aac5e40b60a85dcbc"
format_stamp: "Formatted at 2026-09-25 22:05:37 on dist-test-slave-7mqr"
server_key: "9aec6119736354ca07a5efdb12375443"
server_key_iv: "17e99f26e40e71496c2d42be0d8184f1"
server_key_version: "encryptionkey@0"
I20260925 22:05:37.033705 11547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:37.058337 11547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:37.060388 11547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:37.060544 11547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:37.060799 11547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:37.061102 11547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:37.061151 11547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:37.061187 11547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:37.061214 11547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:37.066953 11547 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.67:32981
I20260925 22:05:37.067039 11675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.67:32981 every 8 connection(s)
I20260925 22:05:37.067360 11547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
I20260925 22:05:37.070035 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 11547
I20260925 22:05:37.070140 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance
I20260925 22:05:37.070344 5121 external_mini_cluster.cc:1428] Setting key b0c64b3359497ee02d8fc5f1381d7e69
Sep 25 22:05:37 dist-test-slave-7mqr krb5kdc[11170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373937, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM
I20260925 22:05:37.076299 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.068095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:41901 (local address 127.5.0.126:41553)
0925 22:05:37.068149 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:37.068151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:37.068357 (+ 206us) server_negotiation.cc:407] Connection header received
0925 22:05:37.068876 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:37.068877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:37.068892 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:37.068937 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:37.069855 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.070622 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.071716 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.071955 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.072021 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.073934 (+ 1913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:37.073935 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:37.073935 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:37.073941 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:37.075147 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.075489 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.075489 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.075490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.075536 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.075733 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.075733 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.075734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.075829 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:37.075879 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:37.076155 (+ 276us) server_negotiation.cc:299] Negotiation successful
0925 22:05:37.076232 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260925 22:05:37.076478 11678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.068106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.5.0.67:41901)
0925 22:05:37.068245 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:37.068278 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:37.068770 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:37.069018 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:37.069021 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:37.069572 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:37.069763 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.069768 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.070719 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.070720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.071642 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.071645 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.072033 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.072034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.072136 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.072552 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:37.072559 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:37.073848 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:37.075219 (+ 1371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.075224 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.075225 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.075429 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.075589 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.075589 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.075590 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.075684 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.075879 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:37.075882 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:37.076035 (+ 153us) client_negotiation.cc:769] Sending connection context
0925 22:05:37.076132 (+ 97us) client_negotiation.cc:240] Negotiation successful
0925 22:05:37.076248 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":73,"threads_started":1}
I20260925 22:05:37.076879 11676 heartbeater.cc:347] Connected to a master server at 127.5.0.126:41553
I20260925 22:05:37.077087 11676 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:37.077262 11676 heartbeater.cc:511] Master 127.5.0.126:41553 requested a full tablet report, sending...
I20260925 22:05:37.077608 11215 ts_manager.cc:194] Registered new tserver with Master: 35c4eb82e71d482aac5e40b60a85dcbc (127.5.0.67:32981)
I20260925 22:05:37.077965 11215 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.0.67@KRBTEST.COM'} at 127.5.0.67:41901
I20260925 22:05:37.081884 5121 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.5.0.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.0.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.0.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.125.keytab.
Entry for principal oryx/127.5.0.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.125.keytab.
I20260925 22:05:37.104529 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.125:40765
--webserver_interface=127.5.0.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.125.keytab
--principal=oryx/127.5.0.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:39513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:41553,127.5.0.125:40765
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:37.183262 11681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:37.183478 11681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:37.183501 11681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:37.184710 11681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:05:37.184752 11681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:37.184767 11681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:37.184815 11681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:05:37.184837 11681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:05:37.186342 11681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:39513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:41553,127.5.0.125:40765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.125.keytab
--principal=oryx/127.5.0.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.125:40765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.0.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
W20260925 22:05:37.186442 11681 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.5.0.126:41553,127.5.0.125:40765'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260925 22:05:37.186635 11681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:37.186868 11681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:37.191550 11686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:37.191526 11689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:37.191519 11687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:37.191794 11681 server_base.cc:1034] running on GCE node
I20260925 22:05:37.192040 11681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:37.192274 11681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:37.193439 11681 hybrid_clock.cc:648] HybridClock initialized: now 1790373937193422 us; error 26 us; skew 500 ppm
Sep 25 22:05:37 dist-test-slave-7mqr krb5kdc[11170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373937, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:37.195851 11681 init.cc:377] Logged in from keytab as oryx/127.5.0.125@KRBTEST.COM (short username oryx)
I20260925 22:05:37.196314 11681 webserver.cc:460] Webserver started at http://127.5.0.125:39899/ using document root <none> and password file <none>
I20260925 22:05:37.196542 11681 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:37.196585 11681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:37.196734 11681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:37.197767 11681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-1/data/instance:
uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f"
format_stamp: "Formatted at 2026-09-25 22:05:37 on dist-test-slave-7mqr"
server_key: "40643abe897db63686282a22ac4e1bbb"
server_key_iv: "2613b482d92073e63cf521af334e84ab"
server_key_version: "encryptionkey@0"
I20260925 22:05:37.198127 11681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-1/wal/instance:
uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f"
format_stamp: "Formatted at 2026-09-25 22:05:37 on dist-test-slave-7mqr"
server_key: "40643abe897db63686282a22ac4e1bbb"
server_key_iv: "2613b482d92073e63cf521af334e84ab"
server_key_version: "encryptionkey@0"
I20260925 22:05:37.199424 11681 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 22:05:37.200327 11696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:37.200487 11681 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:05:37.200563 11681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-1/wal
uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f"
format_stamp: "Formatted at 2026-09-25 22:05:37 on dist-test-slave-7mqr"
server_key: "40643abe897db63686282a22ac4e1bbb"
server_key_iv: "2613b482d92073e63cf521af334e84ab"
server_key_version: "encryptionkey@0"
I20260925 22:05:37.200614 11681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:37.213145 11681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:37.214903 11681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:37.215006 11681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:37.218707 11681 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.125:40765
I20260925 22:05:37.218763 11748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.125:40765 every 8 connection(s)
I20260925 22:05:37.219053 11681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-1/data/info.pb
I20260925 22:05:37.220042 11749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:37.221514 11749 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 22:05:37 dist-test-slave-7mqr krb5kdc[11170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373937, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.125@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM
I20260925 22:05:37.228834 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 11681
I20260925 22:05:37.229919 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.222141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41760 (local address 127.5.0.126:41553)
0925 22:05:37.222220 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:37.222223 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:37.222377 (+ 154us) server_negotiation.cc:407] Connection header received
0925 22:05:37.222821 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:37.222822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:37.222832 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:37.222884 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:37.223421 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.223936 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.224852 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.225026 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.225074 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.227020 (+ 1946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:37.227022 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:37.227022 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:37.227029 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:37.228440 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.228853 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.228854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.228855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.228920 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.229156 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.229157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.229157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.229377 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:37.229451 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:37.229791 (+ 340us) server_negotiation.cc:299] Negotiation successful
0925 22:05:37.229856 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 22:05:37.230197 11751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.222096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.0.0.1:41760)
0925 22:05:37.222269 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:37.222302 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:37.222741 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:37.222941 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:37.222944 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:37.223178 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:37.223355 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.223359 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.224013 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.224014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.224785 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.224787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.225091 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.225092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.225189 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.225583 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:37.225604 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:37.226941 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:37.228511 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.228515 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.228517 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.228781 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.228998 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.228999 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.228999 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.229089 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.229482 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:37.229487 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:37.229640 (+ 153us) client_negotiation.cc:769] Sending connection context
0925 22:05:37.229772 (+ 132us) client_negotiation.cc:240] Negotiation successful
0925 22:05:37.229938 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":97,"threads_started":1}
I20260925 22:05:37.230786 11749 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 22:05:37 dist-test-slave-7mqr krb5kdc[11170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373934, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.0.125@KRBTEST.COM
Sep 25 22:05:37 dist-test-slave-7mqr krb5kdc[11170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373937, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.125@KRBTEST.COM for oryx/127.5.0.125@KRBTEST.COM
I20260925 22:05:37.238554 11753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.229316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59196 (local address 127.5.0.125:40765)
0925 22:05:37.229450 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:37.229453 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:37.229497 (+ 44us) server_negotiation.cc:407] Connection header received
0925 22:05:37.229567 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:37.229569 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:37.229580 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:37.229671 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:37.230389 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.231377 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.232430 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.232869 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.232933 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.235243 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:37.235250 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:37.235250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:37.235256 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:37.237338 (+ 2082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.237719 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.237722 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.237722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.237782 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.237970 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.237971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.237971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.238138 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:37.238244 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:37.238371 (+ 127us) server_negotiation.cc:299] Negotiation successful
0925 22:05:37.238457 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":67,"threads_started":1}
I20260925 22:05:37.239431 11751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.231003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:40765 (local address 127.0.0.1:59198)
0925 22:05:37.231037 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:37.231046 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:37.231075 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:37.231307 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:37.231308 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:37.231454 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:37.231506 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.231507 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.232414 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.232415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.233086 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.233087 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.233510 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.233511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.233557 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.233909 (+ 352us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:37.233913 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:37.235600 (+ 1687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:37.238044 (+ 2444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.238044 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.238045 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.238221 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.238414 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.238415 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.238415 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.238489 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.239107 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:37.239108 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:37.239148 (+ 40us) client_negotiation.cc:769] Sending connection context
0925 22:05:37.239229 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 22:05:37.239322 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260925 22:05:37.239530 11754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.231065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59198 (local address 127.5.0.125:40765)
0925 22:05:37.231188 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:37.231190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:37.231197 (+ 7us) server_negotiation.cc:407] Connection header received
0925 22:05:37.231215 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:37.231216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:37.231229 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:37.231290 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:37.231563 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.232322 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.233154 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.233440 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.233512 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.235686 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:37.235688 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:37.235688 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:37.235695 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:37.237977 (+ 2282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.238290 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.238291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.238291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.238347 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.238640 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.238641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.238642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.238750 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:37.239110 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:37.239251 (+ 141us) server_negotiation.cc:299] Negotiation successful
0925 22:05:37.239374 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":105,"thread_start_us":82,"threads_started":1}
I20260925 22:05:37.240701 11749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f: Bootstrap starting.
I20260925 22:05:37.241269 11749 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:37.241515 11749 log.cc:824] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:37.242188 11749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f: No bootstrap required, opened a new log
I20260925 22:05:37.243506 11749 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } }
I20260925 22:05:37.243625 11749 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:37.243646 11749 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6d92dd7ba964ba6b2f2b1a84b443f9f, State: Initialized, Role: FOLLOWER
I20260925 22:05:37.243764 11749 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } }
I20260925 22:05:37.244079 11749 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:05:37.244037 11756 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } } }
I20260925 22:05:37.244153 11756 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [sys.catalog]: This master's current role is: FOLLOWER
W20260925 22:05:37.245919 11767 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:05:37.246024 11767 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:05:37.247313 11681 master_runner.cc:428] Detected that this master f6d92dd7ba964ba6b2f2b1a84b443f9f is joining an existing cluster
I20260925 22:05:37.247365 11681 master_runner.cc:432] Initiating AddMaster RPC to add 127.5.0.125:40765
I20260925 22:05:37.254487 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.248469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41766 (local address 127.5.0.126:41553)
0925 22:05:37.248513 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:37.248515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:37.248614 (+ 99us) server_negotiation.cc:407] Connection header received
0925 22:05:37.248688 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:37.248689 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:37.248698 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:37.248754 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:37.249095 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.249623 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.250394 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.250602 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.250683 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.251650 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:37.251652 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:37.251652 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:37.251659 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:37.253288 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.253713 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.253714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.253715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.253779 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.253977 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.253978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.253979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.254094 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:37.254166 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:37.254307 (+ 141us) server_negotiation.cc:299] Negotiation successful
0925 22:05:37.254387 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 22:05:37.254683 11774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.248305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.0.0.1:41766)
0925 22:05:37.248541 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:37.248551 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:37.248627 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:37.248758 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:37.248759 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:37.248965 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:37.249023 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.249024 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.249721 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.249722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.250303 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.250305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.250703 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.250704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.250767 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.251226 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:37.251231 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:37.251544 (+ 313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:37.253378 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.253379 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.253379 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.253614 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.253844 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.253845 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.253845 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.253913 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.254156 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:37.254157 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:37.254211 (+ 54us) client_negotiation.cc:769] Sending connection context
0925 22:05:37.254283 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 22:05:37.254392 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":77,"threads_started":1}
I20260925 22:05:37.255932 11754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.248358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59212 (local address 127.5.0.125:40765)
0925 22:05:37.248397 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:37.248399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:37.248620 (+ 221us) server_negotiation.cc:407] Connection header received
0925 22:05:37.248702 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:37.248703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:37.248710 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:37.248736 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:37.249167 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.249899 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.251087 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.251404 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.251447 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.252480 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:37.252481 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:37.252482 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:37.252489 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:37.254174 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.254859 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.254860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.254860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.254917 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.255377 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.255378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.255379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.255504 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:37.255569 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:37.255744 (+ 175us) server_negotiation.cc:299] Negotiation successful
0925 22:05:37.255821 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 22:05:37.256235 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.248400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:40765 (local address 127.0.0.1:59212)
0925 22:05:37.248560 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:37.248570 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:37.248640 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:37.248787 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:37.248787 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:37.249023 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:37.249086 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.249088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.249988 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.249989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.250913 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.250916 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.251515 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.251516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.251572 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.252025 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:37.252030 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:37.252401 (+ 371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:37.254265 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.254266 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.254266 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.254484 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.255244 (+ 760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.255244 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.255245 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.255317 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.255576 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:37.255577 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:37.255620 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 22:05:37.256053 (+ 433us) client_negotiation.cc:240] Negotiation successful
0925 22:05:37.256139 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":90,"threads_started":1}
Sep 25 22:05:37 dist-test-slave-7mqr krb5kdc[11170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373934, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.126@KRBTEST.COM for oryx/127.5.0.125@KRBTEST.COM
I20260925 22:05:37.267089 11775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.257027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:40765 (local address 127.0.0.1:59228)
0925 22:05:37.259988 (+ 2961us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:37.260025 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:37.260091 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:37.260235 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:37.260240 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:37.260485 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:37.260534 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.260535 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.261433 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.261434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.262003 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.262005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.262448 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.262448 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.262504 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.262949 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:37.262953 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:37.264269 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:37.265914 (+ 1645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.265920 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.265921 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.266148 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.266356 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.266356 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.266357 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.266411 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.266687 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:37.266693 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:37.266768 (+ 75us) client_negotiation.cc:769] Sending connection context
0925 22:05:37.266880 (+ 112us) client_negotiation.cc:240] Negotiation successful
0925 22:05:37.266985 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2936,"thread_start_us":71,"threads_started":1}
I20260925 22:05:37.267108 11754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.257075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59228 (local address 127.5.0.125:40765)
0925 22:05:37.257108 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:37.257110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:37.260090 (+ 2980us) server_negotiation.cc:407] Connection header received
0925 22:05:37.260150 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:37.260151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:37.260162 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:37.260224 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:37.260600 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.261347 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.262076 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.262369 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.262434 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.264359 (+ 1925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:37.264361 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:37.264361 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:37.264368 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:37.265844 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.266225 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.266226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.266226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.266286 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.266489 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.266490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.266490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.266617 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:37.266684 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:37.266917 (+ 233us) server_negotiation.cc:299] Negotiation successful
0925 22:05:37.266998 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 22:05:37.267724 11215 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.5.0.125:40765: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f"
member_type: NON_VOTER
last_known_addr {
host: "127.5.0.125"
port: 40765
}
attrs {
promote: true
}
}
dest_uuid: "8faad6f3402b4d1794676034a93d324a"
cas_config_opid_index: -1
I20260925 22:05:37.268074 11215 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: NON_VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: true } }
I20260925 22:05:37.268707 11778 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER f6d92dd7ba964ba6b2f2b1a84b443f9f (127.5.0.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: NON_VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: true } } }
I20260925 22:05:37.268891 11723 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:37.269770 11723 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [term 1 FOLLOWER]: Refusing update from remote peer 8faad6f3402b4d1794676034a93d324a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260925 22:05:37.269783 11778 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.5.0.125:40765
I20260925 22:05:37.269999 11681 master.cc:537] Master@127.5.0.125:40765 shutting down...
I20260925 22:05:37.270088 11777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8faad6f3402b4d1794676034a93d324a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: NON_VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: true } } }
I20260925 22:05:37.270196 11777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:37.270598 11777 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: NON_VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 22:05:37.271456 11200 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260925 22:05:37.272233 11200 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a -> Peer f6d92dd7ba964ba6b2f2b1a84b443f9f (127.5.0.125:40765): Couldn't send request to peer f6d92dd7ba964ba6b2f2b1a84b443f9f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260925 22:05:37.273037 11681 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:05:37.273129 11681 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:37.273180 11681 tablet_replica.cc:330] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f: stopping tablet replica
I20260925 22:05:37.285523 11681 master.cc:552] Master@127.5.0.125:40765 shutdown complete.
I20260925 22:05:37.285576 11681 master_runner.cc:305] Clearing existing system tablet
I20260925 22:05:37.285709 11681 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:37.285763 11681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:37.286414 11681 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:05:37.286900 11782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:37.287052 11681 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:37.287109 11681 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:37.287133 11681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-1/wal
uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f"
format_stamp: "Formatted at 2026-09-25 22:05:37 on dist-test-slave-7mqr"
server_key: "40643abe897db63686282a22ac4e1bbb"
server_key_iv: "2613b482d92073e63cf521af334e84ab"
server_key_version: "encryptionkey@0"
I20260925 22:05:37.288187 11681 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:37.289690 11681 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260925 22:05:37.289757 11681 log.cc:1196] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260925 22:05:37.289954 11681 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f: Deleting consensus metadata
I20260925 22:05:37.290079 11681 master_runner.cc:315] Copying system tablet from 127.5.0.126:41553
I20260925 22:05:37.290692 11681 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f: tablet copy: Beginning tablet copy session from remote peer at address 127.5.0.126:41553
I20260925 22:05:37.296994 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.291040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41780 (local address 127.5.0.126:41553)
0925 22:05:37.291084 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:37.291086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:37.291145 (+ 59us) server_negotiation.cc:407] Connection header received
0925 22:05:37.291220 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:37.291221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:37.291229 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:37.291278 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:37.291615 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.292269 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.293303 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.293480 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.293529 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.294509 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:37.294510 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:37.294510 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:37.294516 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:37.295956 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.296306 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.296307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.296307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.296358 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.296541 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.296542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.296542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.296642 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:37.296688 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:37.296838 (+ 150us) server_negotiation.cc:299] Negotiation successful
0925 22:05:37.296896 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 22:05:37.297155 11788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.290959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.0.0.1:41780)
0925 22:05:37.291076 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:37.291087 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:37.291158 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:37.291284 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:37.291285 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:37.291479 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:37.291545 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.291547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.292333 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.292334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.293223 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.293226 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.293545 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.293546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.293614 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.294027 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:37.294032 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:37.294431 (+ 399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:37.296024 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.296025 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.296026 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.296238 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.296415 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.296415 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.296416 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.296480 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.296697 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:37.296697 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:37.296743 (+ 46us) client_negotiation.cc:769] Sending connection context
0925 22:05:37.296813 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 22:05:37.296933 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":77,"threads_started":1}
I20260925 22:05:37.297472 11235 tablet_copy_service.cc:140] P 8faad6f3402b4d1794676034a93d324a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f6d92dd7ba964ba6b2f2b1a84b443f9f ({username='oryx', principal='oryx/127.5.0.125@KRBTEST.COM'} at 127.0.0.1:41780)
I20260925 22:05:37.297571 11235 tablet_copy_service.cc:161] P 8faad6f3402b4d1794676034a93d324a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f6d92dd7ba964ba6b2f2b1a84b443f9f at {username='oryx', principal='oryx/127.5.0.125@KRBTEST.COM'} at 127.0.0.1:41780: session id = f6d92dd7ba964ba6b2f2b1a84b443f9f-00000000000000000000000000000000
I20260925 22:05:37.298357 11235 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 22:05:37.298918 11681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260925 22:05:37.299800 11681 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f: tablet copy: Starting download of 0 data blocks...
I20260925 22:05:37.299978 11681 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f: tablet copy: Starting download of 1 WAL segments...
I20260925 22:05:37.300774 11681 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 22:05:37.301504 11235 tablet_copy_service.cc:342] P 8faad6f3402b4d1794676034a93d324a: Request end of tablet copy session f6d92dd7ba964ba6b2f2b1a84b443f9f-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.5.0.125@KRBTEST.COM'} at 127.0.0.1:41780
I20260925 22:05:37.301563 11235 tablet_copy_service.cc:433] P 8faad6f3402b4d1794676034a93d324a: ending tablet copy session f6d92dd7ba964ba6b2f2b1a84b443f9f-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f6d92dd7ba964ba6b2f2b1a84b443f9f
I20260925 22:05:37.304018 11681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:37.305135 11789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:37.305181 11792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:37.305190 11790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:37.308063 11681 server_base.cc:1034] running on GCE node
I20260925 22:05:37.308162 11681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:37.308308 11681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:37.309419 11681 hybrid_clock.cc:648] HybridClock initialized: now 1790373937309404 us; error 34 us; skew 500 ppm
Sep 25 22:05:37 dist-test-slave-7mqr krb5kdc[11170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373937, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:37.310653 11681 init.cc:377] Logged in from keytab as oryx/127.5.0.125@KRBTEST.COM (short username oryx)
I20260925 22:05:37.310915 11681 webserver.cc:460] Webserver started at http://127.5.0.125:40435/ using document root <none> and password file <none>
I20260925 22:05:37.311039 11681 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:37.311076 11681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:37.311685 11681 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:05:37.312251 11799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:37.312383 11681 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:37.312431 11681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-1/wal
uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f"
format_stamp: "Formatted at 2026-09-25 22:05:37 on dist-test-slave-7mqr"
server_key: "40643abe897db63686282a22ac4e1bbb"
server_key_iv: "2613b482d92073e63cf521af334e84ab"
server_key_version: "encryptionkey@0"
I20260925 22:05:37.312486 11681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:37.329545 11681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:37.331113 11681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:37.334125 11681 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.125:40765
I20260925 22:05:37.334148 11851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.125:40765 every 8 connection(s)
I20260925 22:05:37.334439 11681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-1/data/info.pb
I20260925 22:05:37.335310 11852 sys_catalog.cc:263] Verifying existing consensus state
I20260925 22:05:37.335841 11852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f: Bootstrap starting.
I20260925 22:05:37.338788 11852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 22:05:37.339038 11852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f: Bootstrap complete.
I20260925 22:05:37.339242 11852 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: NON_VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: true } }
I20260925 22:05:37.339331 11852 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f6d92dd7ba964ba6b2f2b1a84b443f9f, State: Initialized, Role: LEARNER
I20260925 22:05:37.339370 11852 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: NON_VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: true } }
I20260925 22:05:37.339543 11852 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:05:37.339651 11855 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: NON_VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: true } } }
I20260925 22:05:37.339751 11855 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [sys.catalog]: This master's current role is: LEARNER
I20260925 22:05:37.341377 11867 catalog_manager.cc:1259] Loaded cluster ID: 812165beba8f4c9c8ac57ab50e1ed30e
I20260925 22:05:37.341462 11867 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f: loading cluster ID for follower catalog manager: success
I20260925 22:05:37.342680 11867 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f: acquiring CA information for follower catalog manager: success
I20260925 22:05:37.342983 11867 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 22:05:37.347030 11858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.340078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59240 (local address 127.5.0.125:40765)
0925 22:05:37.340213 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:37.340215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:37.340223 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:05:37.340245 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:37.340246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:37.340255 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:37.340352 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:37.340978 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.341993 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.342969 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.343244 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.343326 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.344529 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:37.344531 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:37.344531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:37.344538 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:37.345895 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.346232 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.346233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.346234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.346288 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.346485 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.346486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.346486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.346612 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:37.346723 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:37.346820 (+ 97us) server_negotiation.cc:299] Negotiation successful
0925 22:05:37.346914 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":62,"threads_started":1}
WARNING: no policy specified for oryx/127.5.0.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.0.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.0.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.124.keytab.
Entry for principal oryx/127.5.0.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.124.keytab.
I20260925 22:05:37.371485 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.124:40507
--webserver_interface=127.5.0.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.124.keytab
--principal=oryx/127.5.0.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:39513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:41553,127.5.0.125:40765,127.5.0.124:40507
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:37.453677 11870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:37.453882 11870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:37.453903 11870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:37.455031 11870 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:05:37.455070 11870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:37.455085 11870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:37.455134 11870 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:05:37.455157 11870 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:05:37.456743 11870 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:39513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:41553,127.5.0.125:40765,127.5.0.124:40507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.124.keytab
--principal=oryx/127.5.0.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.124:40507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.0.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:37.456995 11870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:37.457229 11870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:37.461810 11878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:37.461827 11876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:37.461819 11875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:37.462051 11870 server_base.cc:1034] running on GCE node
I20260925 22:05:37.462311 11870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:37.462586 11870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:37.463737 11870 hybrid_clock.cc:648] HybridClock initialized: now 1790373937463729 us; error 37 us; skew 500 ppm
Sep 25 22:05:37 dist-test-slave-7mqr krb5kdc[11170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373937, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:37.466255 11870 init.cc:377] Logged in from keytab as oryx/127.5.0.124@KRBTEST.COM (short username oryx)
I20260925 22:05:37.466645 11870 webserver.cc:460] Webserver started at http://127.5.0.124:43055/ using document root <none> and password file <none>
I20260925 22:05:37.466871 11870 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:37.466948 11870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:37.467056 11870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:37.468143 11870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-2/data/instance:
uuid: "02ee5b2754924933997a4ba119c02639"
format_stamp: "Formatted at 2026-09-25 22:05:37 on dist-test-slave-7mqr"
server_key: "5daedd4a3250c511827bd52235c3a71b"
server_key_iv: "304c26ba27ebaaeb789466e758b75a8a"
server_key_version: "encryptionkey@0"
I20260925 22:05:37.468446 11870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-2/wal/instance:
uuid: "02ee5b2754924933997a4ba119c02639"
format_stamp: "Formatted at 2026-09-25 22:05:37 on dist-test-slave-7mqr"
server_key: "5daedd4a3250c511827bd52235c3a71b"
server_key_iv: "304c26ba27ebaaeb789466e758b75a8a"
server_key_version: "encryptionkey@0"
I20260925 22:05:37.469643 11870 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:05:37.470263 11885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:37.470417 11870 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:37.470511 11870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-2/wal
uuid: "02ee5b2754924933997a4ba119c02639"
format_stamp: "Formatted at 2026-09-25 22:05:37 on dist-test-slave-7mqr"
server_key: "5daedd4a3250c511827bd52235c3a71b"
server_key_iv: "304c26ba27ebaaeb789466e758b75a8a"
server_key_version: "encryptionkey@0"
I20260925 22:05:37.470580 11870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:37.482483 11870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:37.484841 11870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:37.485002 11870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:37.489326 11870 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.124:40507
I20260925 22:05:37.489355 11937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.124:40507 every 8 connection(s)
I20260925 22:05:37.489718 11870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-2/data/info.pb
I20260925 22:05:37.490758 11938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:37.492486 11938 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:05:37.496137 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 11870
Sep 25 22:05:37 dist-test-slave-7mqr krb5kdc[11170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373937, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.124@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM
Sep 25 22:05:37 dist-test-slave-7mqr krb5kdc[11170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373934, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.0.124@KRBTEST.COM
I20260925 22:05:37.501771 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.493218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41790 (local address 127.5.0.126:41553)
0925 22:05:37.493286 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:37.493288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:37.493421 (+ 133us) server_negotiation.cc:407] Connection header received
0925 22:05:37.493907 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:37.493908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:37.493920 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:37.493991 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:37.494612 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.495104 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.496218 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.496407 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.496483 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.498874 (+ 2391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:37.498875 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:37.498875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:37.498887 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:37.500376 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.500814 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.500815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.500816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.500874 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.501116 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.501117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.501117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.501235 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:37.501303 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:37.501600 (+ 297us) server_negotiation.cc:299] Negotiation successful
0925 22:05:37.501680 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 22:05:37.501971 11940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.493092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.0.0.1:41790)
0925 22:05:37.493290 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:37.493336 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:37.493797 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:37.494065 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:37.494074 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:37.494323 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:37.494510 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.494514 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.495198 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.495199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.496118 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.496120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.496514 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.496515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.496684 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.497235 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:37.497243 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:37.498783 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:37.500491 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.500497 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.500499 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.500726 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.500937 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.500938 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.500939 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.501051 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.501305 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:37.501311 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:37.501463 (+ 152us) client_negotiation.cc:769] Sending connection context
0925 22:05:37.501567 (+ 104us) client_negotiation.cc:240] Negotiation successful
0925 22:05:37.501726 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":95,"threads_started":1}
I20260925 22:05:37.503123 11938 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:05:37.505028 11941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.496648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45490 (local address 127.5.0.124:40507)
0925 22:05:37.496786 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:37.496791 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:37.496807 (+ 16us) server_negotiation.cc:407] Connection header received
0925 22:05:37.496832 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:37.496833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:37.496854 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:37.496950 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:37.497496 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.498497 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.499211 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.499508 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.499592 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.501756 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:37.501763 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:37.501763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:37.501770 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:37.503991 (+ 2221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.504343 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.504345 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.504346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.504394 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.504574 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.504575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.504575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.504692 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:37.504827 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:37.504887 (+ 60us) server_negotiation.cc:299] Negotiation successful
0925 22:05:37.504941 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":82,"threads_started":1}
Sep 25 22:05:37 dist-test-slave-7mqr krb5kdc[11170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373937, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.124@KRBTEST.COM for oryx/127.5.0.125@KRBTEST.COM
I20260925 22:05:37.510028 11940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.503384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:40765 (local address 127.0.0.1:59254)
0925 22:05:37.503431 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:37.503442 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:37.503481 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:37.503600 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:37.503600 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:37.503793 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:37.503819 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.503820 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.505035 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.505036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.505672 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.505674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.506093 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.506094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.506151 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.506463 (+ 312us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:37.506467 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:37.507554 (+ 1087us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:37.509042 (+ 1488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.509043 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.509044 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.509206 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.509430 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.509430 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.509431 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.509485 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.509727 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:37.509728 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:37.509769 (+ 41us) client_negotiation.cc:769] Sending connection context
0925 22:05:37.509845 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 22:05:37.509939 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260925 22:05:37.510052 11858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.503458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59254 (local address 127.5.0.125:40765)
0925 22:05:37.503505 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:37.503507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:37.503516 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:05:37.503541 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:37.503542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:37.503550 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:37.503599 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:37.503924 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.504686 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.505767 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.506018 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.506083 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.507639 (+ 1556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:37.507640 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:37.507641 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:37.507648 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:37.508946 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.509287 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.509288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.509289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.509344 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.509551 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.509552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.509552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.509662 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:37.509733 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:37.509883 (+ 150us) server_negotiation.cc:299] Negotiation successful
0925 22:05:37.509961 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 22:05:37.510489 11938 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.0.124" port: 40507 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 22:05:37 dist-test-slave-7mqr krb5kdc[11170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373937, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.124@KRBTEST.COM for oryx/127.5.0.124@KRBTEST.COM
I20260925 22:05:37.516315 11940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.510734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:40507 (local address 127.0.0.1:45496)
0925 22:05:37.510780 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:37.510791 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:37.510833 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:37.510950 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:37.510951 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:37.511072 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:37.511095 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.511096 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.511675 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.511676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.512194 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.512195 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.512593 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.512593 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.512640 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.512929 (+ 289us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:37.512933 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:37.513959 (+ 1026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:37.515283 (+ 1324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.515284 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.515285 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.515439 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.515640 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.515640 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.515641 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.515691 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.515977 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:37.515978 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:37.516014 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 22:05:37.516091 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 22:05:37.516183 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260925 22:05:37.516324 11941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.510805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45496 (local address 127.5.0.124:40507)
0925 22:05:37.510835 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:37.510837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:37.510845 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:05:37.510885 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:37.510885 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:37.510895 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:37.510947 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:37.511174 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.511603 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.512293 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.512516 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.512594 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.514049 (+ 1455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:37.514050 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:37.514050 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:37.514056 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:37.515204 (+ 1148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.515516 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.515517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.515518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.515567 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.515767 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.515767 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.515768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.515864 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:37.515983 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:37.516123 (+ 140us) server_negotiation.cc:299] Negotiation successful
0925 22:05:37.516232 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":17}
I20260925 22:05:37.517601 11938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639: Bootstrap starting.
I20260925 22:05:37.518369 11938 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:37.518587 11938 log.cc:824] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:37.519253 11938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639: No bootstrap required, opened a new log
I20260925 22:05:37.520653 11938 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 40507 } }
I20260925 22:05:37.520784 11938 raft_consensus.cc:374] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:37.520807 11938 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02ee5b2754924933997a4ba119c02639, State: Initialized, Role: FOLLOWER
I20260925 22:05:37.520923 11938 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 40507 } }
I20260925 22:05:37.521170 11938 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:05:37.521160 11943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 40507 } } }
I20260925 22:05:37.521263 11943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 22:05:37.522608 11954 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:05:37.522665 11954 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:05:37.524873 11870 master_runner.cc:428] Detected that this master 02ee5b2754924933997a4ba119c02639 is joining an existing cluster
I20260925 22:05:37.524920 11870 master_runner.cc:432] Initiating AddMaster RPC to add 127.5.0.124:40507
I20260925 22:05:37.532999 11962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.526138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:40507 (local address 127.0.0.1:45510)
0925 22:05:37.526514 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:37.526522 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:37.526586 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:37.526726 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:37.526726 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:37.526990 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:37.527040 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.527041 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.527926 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.527926 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.528788 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.528790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.529264 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.529265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.529323 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.529757 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:37.529762 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:37.530477 (+ 715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:37.531987 (+ 1510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.531988 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.531989 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.532146 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.532342 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.532342 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.532343 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.532396 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.532632 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:37.532633 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:37.532670 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 22:05:37.532763 (+ 93us) client_negotiation.cc:240] Negotiation successful
0925 22:05:37.532871 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":19,"thread_start_us":95,"threads_started":1}
I20260925 22:05:37.533005 11941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.526273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45510 (local address 127.5.0.124:40507)
0925 22:05:37.526310 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:37.526312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:37.526558 (+ 246us) server_negotiation.cc:407] Connection header received
0925 22:05:37.526645 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:37.526646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:37.526654 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:37.526678 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:37.527108 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.527789 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.528874 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.529143 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.529188 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.530585 (+ 1397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:37.530586 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:37.530587 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:37.530593 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:37.531879 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.532227 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.532228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.532229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.532277 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.532459 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.532461 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.532461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.532564 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:37.532630 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:37.532784 (+ 154us) server_negotiation.cc:299] Negotiation successful
0925 22:05:37.532929 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":84,"server-negotiator.queue_time_us":18}
I20260925 22:05:37.548162 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.526027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41792 (local address 127.5.0.126:41553)
0925 22:05:37.526062 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:37.526063 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:37.526339 (+ 276us) server_negotiation.cc:407] Connection header received
0925 22:05:37.526434 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:37.526435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:37.526446 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:37.526489 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:37.526901 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.527710 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.528993 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.529311 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.529354 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.545238 (+ 15884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:37.545240 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:37.545240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:37.545249 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:37.546994 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.547376 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.547377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.547377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.547439 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.547616 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.547616 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.547617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.547763 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:37.547837 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:37.548001 (+ 164us) server_negotiation.cc:299] Negotiation successful
0925 22:05:37.548080 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 22:05:37.548170 11858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.525953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59270 (local address 127.5.0.125:40765)
0925 22:05:37.525993 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:37.525995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:37.526411 (+ 416us) server_negotiation.cc:407] Connection header received
0925 22:05:37.526479 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:37.526480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:37.526492 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:37.526547 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:37.527037 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.527789 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.528915 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.529205 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.529263 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.545135 (+ 15872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:37.545137 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:37.545137 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:37.545145 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:37.546855 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.547307 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.547309 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.547309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.547368 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.547545 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.547546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.547546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.547693 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:37.547765 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:37.547989 (+ 224us) server_negotiation.cc:299] Negotiation successful
0925 22:05:37.548072 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 22:05:37.548332 11960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.525882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:40765 (local address 127.0.0.1:59270)
0925 22:05:37.526351 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:37.526361 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:37.526417 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:37.526591 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:37.526592 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:37.526886 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:37.526953 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.526955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.527873 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.527874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.528839 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.528841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.529305 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.529305 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.529358 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.544583 (+ 15225us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:37.544588 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:37.545018 (+ 430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:37.546952 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.546953 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.546953 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.547226 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.547421 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.547421 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.547422 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.547476 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.547756 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:37.547757 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:37.547804 (+ 47us) client_negotiation.cc:769] Sending connection context
0925 22:05:37.547871 (+ 67us) client_negotiation.cc:240] Negotiation successful
0925 22:05:37.548017 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"mutex_wait_us":77,"thread_start_us":455,"threads_started":1}
I20260925 22:05:37.548511 11961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.525879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.0.0.1:41792)
0925 22:05:37.526276 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:37.526285 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:37.526363 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:37.526556 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:37.526557 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:37.526740 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:37.526811 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.526813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.527831 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.527832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.528890 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.528893 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.529433 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.529434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.529499 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.544530 (+ 15031us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:37.544536 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:37.545085 (+ 549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:37.547075 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.547077 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.547077 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.547310 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.547497 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.547498 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.547499 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.547551 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.547827 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:37.547828 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:37.547863 (+ 35us) client_negotiation.cc:769] Sending connection context
0925 22:05:37.548277 (+ 414us) client_negotiation.cc:240] Negotiation successful
0925 22:05:37.548389 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":65,"thread_start_us":86,"threads_started":1}
Sep 25 22:05:37 dist-test-slave-7mqr krb5kdc[11170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373934, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.126@KRBTEST.COM for oryx/127.5.0.124@KRBTEST.COM
I20260925 22:05:37.556238 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 11277
I20260925 22:05:37.556754 11775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.550583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:40507 (local address 127.0.0.1:45526)
0925 22:05:37.550633 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:37.550643 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:37.550681 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:37.550816 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:37.550817 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:37.550960 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:37.550997 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.550999 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.551592 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.551593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.552268 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.552271 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.552616 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.552617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.552680 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.553055 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:37.553060 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:37.554190 (+ 1130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:37.555567 (+ 1377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.555568 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.555568 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.555763 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.556065 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.556065 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.556066 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.556135 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.556416 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:37.556417 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:37.556456 (+ 39us) client_negotiation.cc:769] Sending connection context
0925 22:05:37.556509 (+ 53us) client_negotiation.cc:240] Negotiation successful
0925 22:05:37.556602 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260925 22:05:37.557081 11941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.550640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45526 (local address 127.5.0.124:40507)
0925 22:05:37.550684 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:37.550686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:37.550694 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:05:37.550747 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:37.550748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:37.550757 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:37.550811 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:37.551067 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.551515 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.552339 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.552539 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.552617 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.554269 (+ 1652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:37.554271 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:37.554271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:37.554278 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:37.555487 (+ 1209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.555930 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.555931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.555931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.555991 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.556213 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.556214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.556219 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.556339 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:37.556393 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:37.556933 (+ 540us) server_negotiation.cc:299] Negotiation successful
0925 22:05:37.556998 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 22:05:37.562865 11215 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.5.0.124:40507: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "02ee5b2754924933997a4ba119c02639"
member_type: NON_VOTER
last_known_addr {
host: "127.5.0.124"
port: 40507
}
attrs {
promote: true
}
}
dest_uuid: "8faad6f3402b4d1794676034a93d324a"
cas_config_opid_index: 5
I20260925 22:05:37.563081 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.65:44879
--local_ip_for_outbound_sockets=127.5.0.65
--tserver_master_addrs=127.5.0.126:41553
--webserver_port=36943
--webserver_interface=127.5.0.65
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.65.keytab
--principal=oryx/127.5.0.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:39513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.0.126:41553,127.5.0.125:40765
--tserver_master_addrs=127.5.0.126:41553,127.5.0.125:40765,127.5.0.124:40507 with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:05:37.563120 11215 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: NON_VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: true } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: NON_VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: true } }
I20260925 22:05:37.563470 11778 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 02ee5b2754924933997a4ba119c02639 (127.5.0.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: NON_VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: true } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: NON_VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: true } } }
I20260925 22:05:37.563759 11912 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:37.564378 11778 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.5.0.124:40507
I20260925 22:05:37.564437 11778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8faad6f3402b4d1794676034a93d324a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: NON_VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: true } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: NON_VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: true } } }
I20260925 22:05:37.564498 11778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:37.564821 11912 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [term 1 FOLLOWER]: Refusing update from remote peer 8faad6f3402b4d1794676034a93d324a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260925 22:05:37.565109 11870 master.cc:537] Master@127.5.0.124:40507 shutting down...
I20260925 22:05:37.565120 11777 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [LEADER]: Connected to new peer: Peer: permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: NON_VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 22:05:37.566159 11199 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a -> Peer 02ee5b2754924933997a4ba119c02639 (127.5.0.124:40507): Couldn't send request to peer 02ee5b2754924933997a4ba119c02639. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260925 22:05:37.569973 11775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.564762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:40765 (local address 127.0.0.1:59272)
0925 22:05:37.564885 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:37.564895 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:37.564929 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:37.565569 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:37.565570 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:37.565572 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:37.565612 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.565613 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.566509 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.566511 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.567787 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.567790 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.569710 (+ 1920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.569711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.569782 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.569808 (+ 26us) client_negotiation.cc:769] Sending connection context
0925 22:05:37.569868 (+ 60us) client_negotiation.cc:240] Negotiation successful
0925 22:05:37.569906 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109}
I20260925 22:05:37.570361 11858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.565004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59272 (local address 127.5.0.125:40765)
0925 22:05:37.565401 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:37.565403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:37.565411 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:05:37.565435 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:37.565436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:37.565458 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:37.565669 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:37.565698 (+ 29us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.566425 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.568933 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.569622 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.569709 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.569796 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:37.570254 (+ 458us) server_negotiation.cc:299] Negotiation successful
0925 22:05:37.570296 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371}
I20260925 22:05:37.570540 11826 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [term 1 LEARNER]: Refusing update from remote peer 8faad6f3402b4d1794676034a93d324a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260925 22:05:37.570712 11777 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: NON_VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260925 22:05:37.570905 11870 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:05:37.571002 11870 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:37.571048 11870 tablet_replica.cc:330] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639: stopping tablet replica
I20260925 22:05:37.571322 11826 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 02ee5b2754924933997a4ba119c02639 (127.5.0.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: NON_VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: true } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: NON_VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: true } } }
I20260925 22:05:37.572600 11855 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8faad6f3402b4d1794676034a93d324a. Latest consensus state: current_term: 1 leader_uuid: "8faad6f3402b4d1794676034a93d324a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: NON_VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: true } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: NON_VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: true } } }
I20260925 22:05:37.572711 11855 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [sys.catalog]: This master's current role is: LEARNER
I20260925 22:05:37.572762 11855 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8faad6f3402b4d1794676034a93d324a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: NON_VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: true } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: NON_VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: true } } }
I20260925 22:05:37.572835 11855 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [sys.catalog]: This master's current role is: LEARNER
I20260925 22:05:37.576404 11777 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a: attempting to promote NON_VOTER f6d92dd7ba964ba6b2f2b1a84b443f9f to VOTER
I20260925 22:05:37.576602 11777 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: false } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: NON_VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: true } }
I20260925 22:05:37.577073 11775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.576972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:40507 (local address 127.0.0.1:45528)
0925 22:05:37.577010 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:37.577045 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.124:40507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260925 22:05:37.577121 11826 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [term 1 LEARNER]: Refusing update from remote peer 8faad6f3402b4d1794676034a93d324a: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260925 22:05:37.577291 11777 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260925 22:05:37.577497 11775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.577371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:40507 (local address 127.0.0.1:45530)
0925 22:05:37.577435 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:37.577457 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.124:40507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260925 22:05:37.577628 11199 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a -> Peer 02ee5b2754924933997a4ba119c02639 (127.5.0.124:40507): Couldn't send request to peer 02ee5b2754924933997a4ba119c02639. Status: Network error: Client connection negotiation failed: client connection to 127.5.0.124:40507: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 22:05:37.577971 11778 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, f6d92dd7ba964ba6b2f2b1a84b443f9f (127.5.0.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: false } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: NON_VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: true } } }
I20260925 22:05:37.578810 11778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8faad6f3402b4d1794676034a93d324a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: false } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: NON_VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: true } } }
I20260925 22:05:37.578886 11778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:37.578930 11778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8faad6f3402b4d1794676034a93d324a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: false } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: NON_VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: true } } }
I20260925 22:05:37.578975 11778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:37.579191 11826 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, f6d92dd7ba964ba6b2f2b1a84b443f9f (127.5.0.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: false } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: NON_VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: true } } }
I20260925 22:05:37.580327 11965 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8faad6f3402b4d1794676034a93d324a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: false } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: NON_VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: true } } }
I20260925 22:05:37.580474 11965 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:05:37.584868 11870 master.cc:552] Master@127.5.0.124:40507 shutdown complete.
I20260925 22:05:37.584916 11870 master_runner.cc:305] Clearing existing system tablet
I20260925 22:05:37.585053 11870 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:37.585098 11870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:37.585793 11870 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.001s
I20260925 22:05:37.586378 11968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:37.586544 11870 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:37.586604 11870 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:37.586628 11870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-2/wal
uuid: "02ee5b2754924933997a4ba119c02639"
format_stamp: "Formatted at 2026-09-25 22:05:37 on dist-test-slave-7mqr"
server_key: "5daedd4a3250c511827bd52235c3a71b"
server_key_iv: "304c26ba27ebaaeb789466e758b75a8a"
server_key_version: "encryptionkey@0"
I20260925 22:05:37.587395 11870 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:37.588908 11870 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260925 22:05:37.588974 11870 log.cc:1196] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260925 22:05:37.589200 11870 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639: Deleting consensus metadata
I20260925 22:05:37.589321 11870 master_runner.cc:315] Copying system tablet from 127.5.0.126:41553
I20260925 22:05:37.589898 11870 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639: tablet copy: Beginning tablet copy session from remote peer at address 127.5.0.126:41553
I20260925 22:05:37.597388 11974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.590176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.0.0.1:41794)
0925 22:05:37.590292 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:37.590303 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:37.590360 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:37.590503 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:37.590504 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:37.590696 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:37.590750 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.590752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.591683 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.591685 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.592519 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.592522 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.592977 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.592978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.593036 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.593988 (+ 952us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:37.593994 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:37.594381 (+ 387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:37.596198 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.596200 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.596200 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.596481 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.596735 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.596736 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.596736 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.596801 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.597065 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:37.597066 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:37.597112 (+ 46us) client_negotiation.cc:769] Sending connection context
0925 22:05:37.597181 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 22:05:37.597282 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":65,"threads_started":1}
I20260925 22:05:37.597386 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.590249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41794 (local address 127.5.0.126:41553)
0925 22:05:37.590295 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:37.590297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:37.590347 (+ 50us) server_negotiation.cc:407] Connection header received
0925 22:05:37.590412 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:37.590413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:37.590423 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:37.590495 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:37.590823 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.591598 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.592601 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.592900 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.592971 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.594475 (+ 1504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:37.594477 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:37.594477 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:37.594484 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:37.596098 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.596602 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.596603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.596604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.596665 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.596865 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.596866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.596866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.596996 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:37.597063 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:37.597220 (+ 157us) server_negotiation.cc:299] Negotiation successful
0925 22:05:37.597294 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 22:05:37.597995 11235 tablet_copy_service.cc:140] P 8faad6f3402b4d1794676034a93d324a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 02ee5b2754924933997a4ba119c02639 ({username='oryx', principal='oryx/127.5.0.124@KRBTEST.COM'} at 127.0.0.1:41794)
I20260925 22:05:37.598055 11235 tablet_copy_service.cc:161] P 8faad6f3402b4d1794676034a93d324a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 02ee5b2754924933997a4ba119c02639 at {username='oryx', principal='oryx/127.5.0.124@KRBTEST.COM'} at 127.0.0.1:41794: session id = 02ee5b2754924933997a4ba119c02639-00000000000000000000000000000000
I20260925 22:05:37.598611 11235 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 22:05:37.599037 11870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260925 22:05:37.600284 11870 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639: tablet copy: Starting download of 0 data blocks...
I20260925 22:05:37.600422 11870 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639: tablet copy: Starting download of 1 WAL segments...
I20260925 22:05:37.601444 11870 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 22:05:37.602264 11235 tablet_copy_service.cc:342] P 8faad6f3402b4d1794676034a93d324a: Request end of tablet copy session 02ee5b2754924933997a4ba119c02639-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.5.0.124@KRBTEST.COM'} at 127.0.0.1:41794
I20260925 22:05:37.602319 11235 tablet_copy_service.cc:433] P 8faad6f3402b4d1794676034a93d324a: ending tablet copy session 02ee5b2754924933997a4ba119c02639-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 02ee5b2754924933997a4ba119c02639
I20260925 22:05:37.605188 11870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:37.606211 11975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:37.608857 11976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:37.609071 11978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:37.609207 11870 server_base.cc:1034] running on GCE node
I20260925 22:05:37.609324 11870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:37.609453 11870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:37.610582 11870 hybrid_clock.cc:648] HybridClock initialized: now 1790373937610580 us; error 59 us; skew 500 ppm
Sep 25 22:05:37 dist-test-slave-7mqr krb5kdc[11170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373937, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:37.612131 11870 init.cc:377] Logged in from keytab as oryx/127.5.0.124@KRBTEST.COM (short username oryx)
I20260925 22:05:37.612408 11870 webserver.cc:460] Webserver started at http://127.5.0.124:33535/ using document root <none> and password file <none>
I20260925 22:05:37.612529 11870 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:37.612568 11870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:37.613286 11870 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:05:37.613833 11985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:37.614022 11870 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:05:37.614079 11870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-2/wal
uuid: "02ee5b2754924933997a4ba119c02639"
format_stamp: "Formatted at 2026-09-25 22:05:37 on dist-test-slave-7mqr"
server_key: "5daedd4a3250c511827bd52235c3a71b"
server_key_iv: "304c26ba27ebaaeb789466e758b75a8a"
server_key_version: "encryptionkey@0"
I20260925 22:05:37.614135 11870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:37.633424 11870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:37.635216 11870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:37.638201 11870 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.124:40507
I20260925 22:05:37.638530 11870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/master-2/data/info.pb
I20260925 22:05:37.639504 12038 sys_catalog.cc:263] Verifying existing consensus state
I20260925 22:05:37.640353 12037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.124:40507 every 8 connection(s)
I20260925 22:05:37.640409 12038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639: Bootstrap starting.
I20260925 22:05:37.643363 12038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 22:05:37.643579 12038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639: Bootstrap complete.
I20260925 22:05:37.643795 12038 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: false } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: NON_VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: true } }
I20260925 22:05:37.643880 12038 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 02ee5b2754924933997a4ba119c02639, State: Initialized, Role: LEARNER
I20260925 22:05:37.643944 12038 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: false } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: NON_VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: true } }
I20260925 22:05:37.644140 12038 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:05:37.644148 12041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: false } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: NON_VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: true } } }
I20260925 22:05:37.644438 12041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [sys.catalog]: This master's current role is: LEARNER
I20260925 22:05:37.645628 12052 catalog_manager.cc:1259] Loaded cluster ID: 812165beba8f4c9c8ac57ab50e1ed30e
I20260925 22:05:37.645714 12052 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639: loading cluster ID for follower catalog manager: success
I20260925 22:05:37.646970 12052 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639: acquiring CA information for follower catalog manager: success
I20260925 22:05:37.647324 12052 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260925 22:05:37.666623 11963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:37.666867 11963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:37.666896 11963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:37.668236 11963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:37.668278 11963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:37.668306 11963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.65
I20260925 22:05:37.669778 11963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:39513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.65.keytab
--principal=oryx/127.5.0.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.65:44879
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.0.65
--webserver_port=36943
--tserver_master_addrs=127.5.0.126:41553,127.5.0.125:40765,127.5.0.124:40507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.65
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:37.670013 11963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:37.670258 11963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:37.676020 12058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:37.676155 12057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:37.676014 12060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:37.676416 11963 server_base.cc:1034] running on GCE node
I20260925 22:05:37.676627 11963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:37.676882 11963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:37.678059 11963 hybrid_clock.cc:648] HybridClock initialized: now 1790373937678017 us; error 46 us; skew 500 ppm
Sep 25 22:05:37 dist-test-slave-7mqr krb5kdc[11170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373937, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:37.680620 11963 init.cc:377] Logged in from keytab as oryx/127.5.0.65@KRBTEST.COM (short username oryx)
I20260925 22:05:37.681100 11963 webserver.cc:460] Webserver started at http://127.5.0.65:36943/ using document root <none> and password file <none>
I20260925 22:05:37.681325 11963 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:37.681404 11963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:37.682796 11963 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:05:37.683409 12067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:37.683669 11963 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:05:37.683786 11963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-0/wal
uuid: "2d23d1eb9a374b9a87383727acd62f84"
format_stamp: "Formatted at 2026-09-25 22:05:35 on dist-test-slave-7mqr"
server_key: "2cb7446d9026e315dcbd725ef5828945"
server_key_iv: "3e78d4696cadbfb72c4ce3db99833482"
server_key_version: "encryptionkey@0"
I20260925 22:05:37.684223 11963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:37.708110 11963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:37.710214 11963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:37.710378 11963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:37.710616 11963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:37.710930 11963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:37.710980 11963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:37.711021 11963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:37.711050 11963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:37.717585 11963 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.65:44879
I20260925 22:05:37.717628 12180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.65:44879 every 8 connection(s)
I20260925 22:05:37.718000 11963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
I20260925 22:05:37.719805 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 11963
I20260925 22:05:37.719985 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 11412
Sep 25 22:05:37 dist-test-slave-7mqr krb5kdc[11170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373937, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.125@KRBTEST.COM
Sep 25 22:05:37 dist-test-slave-7mqr krb5kdc[11170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373937, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM
I20260925 22:05:37.730002 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.718773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:33429 (local address 127.5.0.126:41553)
0925 22:05:37.718823 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:37.718825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:37.720335 (+ 1510us) server_negotiation.cc:407] Connection header received
0925 22:05:37.720417 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:37.720418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:37.720429 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:37.720455 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:37.721005 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.721505 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.722549 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.722755 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.722837 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.727037 (+ 4200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:37.727038 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:37.727039 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:37.727045 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:37.728636 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.729006 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.729006 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.729007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.729052 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.729249 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.729250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.729250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.729365 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:37.729439 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:37.729778 (+ 339us) server_negotiation.cc:299] Negotiation successful
0925 22:05:37.729873 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 22:05:37.730010 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.66:34921
--local_ip_for_outbound_sockets=127.5.0.66
--tserver_master_addrs=127.5.0.126:41553
--webserver_port=36691
--webserver_interface=127.5.0.66
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.66.keytab
--principal=oryx/127.5.0.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:39513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.0.126:41553,127.5.0.125:40765
--tserver_master_addrs=127.5.0.126:41553,127.5.0.125:40765,127.5.0.124:40507 with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:05:37.730064 11858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.719010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:39901 (local address 127.5.0.125:40765)
0925 22:05:37.719078 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:37.719080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:37.719172 (+ 92us) server_negotiation.cc:407] Connection header received
0925 22:05:37.719667 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:37.719668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:37.719677 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:37.719728 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:37.721019 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.721783 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.723009 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.723278 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.723358 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.726345 (+ 2987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:37.726346 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:37.726346 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:37.726366 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:37.728099 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.728582 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.728583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.728583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.728645 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.729469 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.729470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.729470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.729602 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:37.729674 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:37.729862 (+ 188us) server_negotiation.cc:299] Negotiation successful
0925 22:05:37.729956 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 22:05:37.730590 12186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.718715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:40765 (local address 127.5.0.65:39901)
0925 22:05:37.719043 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:37.719088 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:37.719566 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:37.719801 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:37.719805 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:37.720697 (+ 892us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:37.720912 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.720918 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.721879 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.721880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.722925 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.722927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.723357 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.723358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.723421 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.723804 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:37.723809 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:37.726257 (+ 2448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:37.728167 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.728173 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.728175 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.728502 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.729047 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.729048 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.729048 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.729182 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.729672 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:37.729673 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:37.729723 (+ 50us) client_negotiation.cc:769] Sending connection context
0925 22:05:37.730192 (+ 469us) client_negotiation.cc:240] Negotiation successful
0925 22:05:37.730360 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":3,"thread_start_us":100,"threads_started":1}
I20260925 22:05:37.731114 12183 heartbeater.cc:347] Connected to a master server at 127.5.0.125:40765
I20260925 22:05:37.731223 12183 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:37.731438 12183 heartbeater.cc:511] Master 127.5.0.125:40765 requested a full tablet report, sending...
I20260925 22:05:37.731855 12187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.718714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.5.0.65:33429)
0925 22:05:37.720289 (+ 1575us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:37.720298 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:37.720356 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:37.720509 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:37.720510 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:37.720724 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:37.720912 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.720916 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.721614 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.721615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.722453 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.722455 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.722843 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.722845 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.722990 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.723635 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:37.723643 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:37.726946 (+ 3303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:37.728718 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.728718 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.728719 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.728938 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.729125 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.729126 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.729126 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.729187 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.729429 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:37.729435 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:37.729600 (+ 165us) client_negotiation.cc:769] Sending connection context
0925 22:05:37.731614 (+ 2014us) client_negotiation.cc:240] Negotiation successful
0925 22:05:37.731734 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1541,"mutex_wait_us":41,"thread_start_us":79,"threads_started":1}
I20260925 22:05:37.732046 11816 ts_manager.cc:194] Registered new tserver with Master: 2d23d1eb9a374b9a87383727acd62f84 (127.5.0.65:44879)
I20260925 22:05:37.732235 12182 heartbeater.cc:347] Connected to a master server at 127.5.0.126:41553
I20260925 22:05:37.732280 12182 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:37.732337 12182 heartbeater.cc:511] Master 127.5.0.126:41553 requested a full tablet report, sending...
I20260925 22:05:37.732605 11215 ts_manager.cc:194] Re-registered known tserver with Master: 2d23d1eb9a374b9a87383727acd62f84 (127.5.0.65:44879)
I20260925 22:05:37.733106 11215 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.0.65@KRBTEST.COM'} at 127.5.0.65:33429
Sep 25 22:05:37 dist-test-slave-7mqr krb5kdc[11170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373937, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.124@KRBTEST.COM
I20260925 22:05:37.737250 12188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.718993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:57261 (local address 127.5.0.124:40507)
0925 22:05:37.719118 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:37.719120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:37.729759 (+ 10639us) server_negotiation.cc:407] Connection header received
0925 22:05:37.729790 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:37.729791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:37.729809 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:37.730250 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:37.730291 (+ 41us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.731227 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.732133 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.732447 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.732938 (+ 491us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.734622 (+ 1684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:37.734623 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:37.734624 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:37.734631 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:37.736080 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.736482 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.736483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.736484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.736542 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.736733 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.736734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.736734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.736846 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:37.736914 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:37.737068 (+ 154us) server_negotiation.cc:299] Negotiation successful
0925 22:05:37.737150 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":74,"threads_started":1}
I20260925 22:05:37.737622 12185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.718714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:40507 (local address 127.5.0.65:57261)
0925 22:05:37.720016 (+ 1302us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:37.720038 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:37.720101 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:37.729915 (+ 9814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:37.729916 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:37.730098 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:37.730173 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.730176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.731312 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.731313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.732049 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.732052 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.732532 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.732532 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.732609 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.733041 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:37.733045 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:37.734534 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:37.736168 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.736168 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.736169 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.736342 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.736606 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.736607 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.736607 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.736664 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.736907 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:37.736908 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:37.736959 (+ 51us) client_negotiation.cc:769] Sending connection context
0925 22:05:37.737045 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 22:05:37.737149 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1272,"mutex_wait_us":3,"thread_start_us":95,"threads_started":1}
I20260925 22:05:37.738039 12181 heartbeater.cc:347] Connected to a master server at 127.5.0.124:40507
I20260925 22:05:37.738095 12181 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:37.738170 12181 heartbeater.cc:511] Master 127.5.0.124:40507 requested a full tablet report, sending...
I20260925 22:05:37.738497 11993 ts_manager.cc:194] Registered new tserver with Master: 2d23d1eb9a374b9a87383727acd62f84 (127.5.0.65:44879)
W20260925 22:05:37.815235 12189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:37.815448 12189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:37.815471 12189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:37.816730 12189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:37.816772 12189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:37.816820 12189 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.66
I20260925 22:05:37.818252 12189 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:39513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.66.keytab
--principal=oryx/127.5.0.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.66:34921
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.0.66
--webserver_port=36691
--tserver_master_addrs=127.5.0.126:41553,127.5.0.125:40765,127.5.0.124:40507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.66
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:37.818504 12189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:37.818747 12189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:37.823560 12195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:37.823554 12197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:37.823805 12189 server_base.cc:1034] running on GCE node
W20260925 22:05:37.823554 12194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:37.824049 12189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:37.824244 12189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:37.825394 12189 hybrid_clock.cc:648] HybridClock initialized: now 1790373937825377 us; error 30 us; skew 500 ppm
Sep 25 22:05:37 dist-test-slave-7mqr krb5kdc[11170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373937, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:37.828145 12189 init.cc:377] Logged in from keytab as oryx/127.5.0.66@KRBTEST.COM (short username oryx)
I20260925 22:05:37.828579 12189 webserver.cc:460] Webserver started at http://127.5.0.66:36691/ using document root <none> and password file <none>
I20260925 22:05:37.828778 12189 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:37.828831 12189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:37.830068 12189 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:05:37.830605 12204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:37.830776 12189 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:37.830857 12189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-1/wal
uuid: "e74b9533a2084cb5a37834fc583b7beb"
format_stamp: "Formatted at 2026-09-25 22:05:36 on dist-test-slave-7mqr"
server_key: "e83eaa054ee9d5dfc73bea8acf6cd53f"
server_key_iv: "7e57ef5e2f986417aa3ef2d533d9aca3"
server_key_version: "encryptionkey@0"
I20260925 22:05:37.831161 12189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:37.854193 12189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:37.856104 12189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:37.856248 12189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:37.856459 12189 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:37.856758 12189 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:37.856791 12189 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:37.856832 12189 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:37.856858 12189 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:37.862525 12189 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.66:34921
I20260925 22:05:37.862592 12317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.66:34921 every 8 connection(s)
I20260925 22:05:37.862932 12189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
I20260925 22:05:37.866572 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 12189
I20260925 22:05:37.866745 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 11547
Sep 25 22:05:37 dist-test-slave-7mqr krb5kdc[11170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373937, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.124@KRBTEST.COM
Sep 25 22:05:37 dist-test-slave-7mqr krb5kdc[11170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373937, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.125@KRBTEST.COM
Sep 25 22:05:37 dist-test-slave-7mqr krb5kdc[11170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373937, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM
I20260925 22:05:37.874562 12188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.863855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:49605 (local address 127.5.0.124:40507)
0925 22:05:37.863931 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:37.863933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:37.863945 (+ 12us) server_negotiation.cc:407] Connection header received
0925 22:05:37.864610 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:37.864612 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:37.864627 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:37.864658 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:37.865551 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.866122 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.867781 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.868125 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.868167 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.871109 (+ 2942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:37.871111 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:37.871111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:37.871119 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:37.872904 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.873398 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.873400 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.873400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.873458 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.873783 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.873784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.873785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.873916 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:37.873961 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:37.874352 (+ 391us) server_negotiation.cc:299] Negotiation successful
0925 22:05:37.874439 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 22:05:37.875363 12322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.863642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:40507 (local address 127.5.0.66:49605)
0925 22:05:37.863812 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:37.863853 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:37.864488 (+ 635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:37.864737 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:37.864742 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:37.865261 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:37.865448 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.865454 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.866222 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.866223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.867689 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.867690 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.868264 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.868265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.868403 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.869021 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:37.869031 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:37.871017 (+ 1986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:37.873001 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.873007 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.873009 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.873299 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.873556 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.873557 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.873557 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.873698 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.874024 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:37.874031 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:37.874196 (+ 165us) client_negotiation.cc:769] Sending connection context
0925 22:05:37.874911 (+ 715us) client_negotiation.cc:240] Negotiation successful
0925 22:05:37.875097 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":59,"thread_start_us":97,"threads_started":1}
I20260925 22:05:37.875496 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.863638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:42039 (local address 127.5.0.126:41553)
0925 22:05:37.863689 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:37.863692 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:37.864208 (+ 516us) server_negotiation.cc:407] Connection header received
0925 22:05:37.864611 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:37.864612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:37.864625 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:37.864658 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:37.865569 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.866080 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.867801 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.868152 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.868196 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.872408 (+ 4212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:37.872409 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:37.872410 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:37.872418 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:37.874242 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.874665 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.874666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.874666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.874732 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.874920 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.874920 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.874921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.875040 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:37.875085 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:37.875289 (+ 204us) server_negotiation.cc:299] Negotiation successful
0925 22:05:37.875366 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 22:05:37.875622 11858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.864012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:57171 (local address 127.5.0.125:40765)
0925 22:05:37.864058 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:37.864059 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:37.864278 (+ 219us) server_negotiation.cc:407] Connection header received
0925 22:05:37.864581 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:37.864582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:37.864594 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:37.864649 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:37.866225 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.866925 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.868503 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.868754 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.868813 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.872378 (+ 3565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:37.872379 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:37.872379 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:37.872387 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:37.874344 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.874801 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.874802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.874803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.874857 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:37.875046 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:37.875046 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:37.875047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:37.875176 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:37.875214 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:37.875407 (+ 193us) server_negotiation.cc:299] Negotiation successful
0925 22:05:37.875503 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 22:05:37.875809 12323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.863642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.5.0.66:42039)
0925 22:05:37.864135 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:37.864144 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:37.864494 (+ 350us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:37.864737 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:37.864742 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:37.865253 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:37.865448 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.865454 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.866197 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.866198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.867690 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.867693 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.868271 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.868271 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.868404 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.869043 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:37.869048 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:37.872294 (+ 3246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:37.874323 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.874324 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.874324 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.874575 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.874804 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.874805 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.874805 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.874858 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.875133 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:37.875134 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:37.875170 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 22:05:37.875567 (+ 397us) client_negotiation.cc:240] Negotiation successful
0925 22:05:37.875672 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"mutex_wait_us":82,"spinlock_wait_cycles":7552,"thread_start_us":94,"threads_started":1}
I20260925 22:05:37.876010 12324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:37.864005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:40765 (local address 127.5.0.66:57171)
0925 22:05:37.864222 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:37.864232 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:37.864496 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:37.865010 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:37.865011 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:37.865250 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:37.865446 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.865451 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.867598 (+ 2147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.867599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.868409 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:37.868411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:37.868887 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:37.868887 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:37.868946 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:37.869303 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:37.869308 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:37.872280 (+ 2972us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:37.874429 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.874430 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.874431 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.874721 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.874924 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:37.874925 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:37.874925 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:37.874983 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:37.875259 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:37.875260 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:37.875303 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 22:05:37.875691 (+ 388us) client_negotiation.cc:240] Negotiation successful
0925 22:05:37.875805 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":1,"thread_start_us":66,"threads_started":1}
I20260925 22:05:37.876394 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.67:32981
--local_ip_for_outbound_sockets=127.5.0.67
--tserver_master_addrs=127.5.0.126:41553
--webserver_port=37713
--webserver_interface=127.5.0.67
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.67.keytab
--principal=oryx/127.5.0.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:39513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.0.126:41553,127.5.0.125:40765
--tserver_master_addrs=127.5.0.126:41553,127.5.0.125:40765,127.5.0.124:40507 with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:05:37.876564 12318 heartbeater.cc:347] Connected to a master server at 127.5.0.124:40507
I20260925 22:05:37.876647 12318 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:37.876734 12320 heartbeater.cc:347] Connected to a master server at 127.5.0.125:40765
I20260925 22:05:37.876788 12320 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:37.876828 12318 heartbeater.cc:511] Master 127.5.0.124:40507 requested a full tablet report, sending...
I20260925 22:05:37.876857 12320 heartbeater.cc:511] Master 127.5.0.125:40765 requested a full tablet report, sending...
I20260925 22:05:37.877285 11993 ts_manager.cc:194] Registered new tserver with Master: e74b9533a2084cb5a37834fc583b7beb (127.5.0.66:34921)
I20260925 22:05:37.877337 11816 ts_manager.cc:194] Registered new tserver with Master: e74b9533a2084cb5a37834fc583b7beb (127.5.0.66:34921)
I20260925 22:05:37.877918 12319 heartbeater.cc:347] Connected to a master server at 127.5.0.126:41553
I20260925 22:05:37.877985 12319 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:37.878051 12319 heartbeater.cc:511] Master 127.5.0.126:41553 requested a full tablet report, sending...
I20260925 22:05:37.878249 11215 ts_manager.cc:194] Re-registered known tserver with Master: e74b9533a2084cb5a37834fc583b7beb (127.5.0.66:34921)
I20260925 22:05:37.878762 11215 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.0.66@KRBTEST.COM'} at 127.5.0.66:42039
W20260925 22:05:37.959450 12325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:37.959724 12325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:37.959749 12325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:37.960943 12325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:37.960980 12325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:37.961009 12325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.67
I20260925 22:05:37.962396 12325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:39513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.67.keytab
--principal=oryx/127.5.0.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.67:32981
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.0.67
--webserver_port=37713
--tserver_master_addrs=127.5.0.126:41553,127.5.0.125:40765,127.5.0.124:40507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.67
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:37.962579 12325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:37.962747 12325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:37.967437 12331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:37.967484 12333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:37.967446 12330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:37.967792 12325 server_base.cc:1034] running on GCE node
I20260925 22:05:37.968030 12325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:37.968271 12325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:37.969424 12325 hybrid_clock.cc:648] HybridClock initialized: now 1790373937969421 us; error 34 us; skew 500 ppm
Sep 25 22:05:37 dist-test-slave-7mqr krb5kdc[11170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373937, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:37.971951 12325 init.cc:377] Logged in from keytab as oryx/127.5.0.67@KRBTEST.COM (short username oryx)
I20260925 22:05:37.972436 12325 webserver.cc:460] Webserver started at http://127.5.0.67:37713/ using document root <none> and password file <none>
I20260925 22:05:37.972671 12325 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:37.972749 12325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:37.974054 12325 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:05:37.974586 12340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:37.974742 12325 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:37.974833 12325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-2/wal
uuid: "35c4eb82e71d482aac5e40b60a85dcbc"
format_stamp: "Formatted at 2026-09-25 22:05:37 on dist-test-slave-7mqr"
server_key: "9aec6119736354ca07a5efdb12375443"
server_key_iv: "17e99f26e40e71496c2d42be0d8184f1"
server_key_version: "encryptionkey@0"
I20260925 22:05:37.975188 12325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:38.000618 12325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:38.002820 12325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:38.002978 12325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:38.003224 12325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:38.003532 12325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:38.003582 12325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:38.003624 12325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:38.003659 12325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:38.010025 12325 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.67:32981
I20260925 22:05:38.010069 12453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.67:32981 every 8 connection(s)
I20260925 22:05:38.010430 12325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
I20260925 22:05:38.011554 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 12325
Sep 25 22:05:38 dist-test-slave-7mqr krb5kdc[11170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373937, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.125@KRBTEST.COM
Sep 25 22:05:38 dist-test-slave-7mqr krb5kdc[11170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373937, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.124@KRBTEST.COM
Sep 25 22:05:38 dist-test-slave-7mqr krb5kdc[11170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373937, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM
I20260925 22:05:38.023416 12188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:38.011219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:47491 (local address 127.5.0.124:40507)
0925 22:05:38.011266 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:38.011272 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:38.011642 (+ 370us) server_negotiation.cc:407] Connection header received
0925 22:05:38.012384 (+ 742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:38.012386 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:38.012398 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:38.012428 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:38.013879 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.014632 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.016145 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.016430 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.016470 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:38.019978 (+ 3508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:38.019979 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:38.019979 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:38.019986 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:38.022232 (+ 2246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:38.022602 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:38.022603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:38.022603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:38.022660 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:38.022851 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:38.022852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:38.022852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:38.022968 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:38.023023 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:38.023217 (+ 194us) server_negotiation.cc:299] Negotiation successful
0925 22:05:38.023297 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 22:05:38.023756 12460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:38.011140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:40765 (local address 127.5.0.67:54415)
0925 22:05:38.011592 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:38.011603 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:38.012234 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:38.012895 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:38.012896 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:38.013199 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:38.013662 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:38.013669 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.014689 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.014690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:38.016074 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:38.016076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.016552 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.016552 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:38.016693 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:38.017441 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:38.017458 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:38.019618 (+ 2160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:38.021867 (+ 2249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:38.021874 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:38.021876 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:38.022197 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:38.022418 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:38.022419 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:38.022419 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:38.022568 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:38.022863 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:38.022868 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:38.023048 (+ 180us) client_negotiation.cc:769] Sending connection context
0925 22:05:38.023156 (+ 108us) client_negotiation.cc:240] Negotiation successful
0925 22:05:38.023325 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"mutex_wait_us":59,"thread_start_us":72,"threads_started":1}
I20260925 22:05:38.023763 12459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:38.011140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:40507 (local address 127.5.0.67:47491)
0925 22:05:38.011509 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:38.011551 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:38.012225 (+ 674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:38.012555 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:38.012559 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:38.013434 (+ 875us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:38.013793 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:38.013794 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.014717 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.014718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:38.016048 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:38.016051 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.016551 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.016552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:38.016693 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:38.018255 (+ 1562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:38.018260 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:38.019859 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:38.022307 (+ 2448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:38.022308 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:38.022308 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:38.022522 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:38.022728 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:38.022728 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:38.022729 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:38.022781 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:38.023057 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:38.023058 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:38.023100 (+ 42us) client_negotiation.cc:769] Sending connection context
0925 22:05:38.023549 (+ 449us) client_negotiation.cc:240] Negotiation successful
0925 22:05:38.023641 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":63,"thread_start_us":86,"threads_started":1}
I20260925 22:05:38.024070 11858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:38.011401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:54415 (local address 127.5.0.125:40765)
0925 22:05:38.011451 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:38.011453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:38.011667 (+ 214us) server_negotiation.cc:407] Connection header received
0925 22:05:38.012398 (+ 731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:38.012399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:38.012412 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:38.012461 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:38.013761 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.014566 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.016158 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.016440 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.016503 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:38.019717 (+ 3214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:38.019718 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:38.019718 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:38.019726 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:38.021777 (+ 2051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:38.022280 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:38.022281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:38.022281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:38.022345 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:38.022645 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:38.022645 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:38.022646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:38.022775 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:38.022837 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:38.023653 (+ 816us) server_negotiation.cc:299] Negotiation successful
0925 22:05:38.023726 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 22:05:38.024225 12468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:38.012779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45536 (local address 127.5.0.124:40507)
0925 22:05:38.013016 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:38.013018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:38.013217 (+ 199us) server_negotiation.cc:407] Connection header received
0925 22:05:38.013245 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:38.013246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:38.013258 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:38.013310 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:38.014619 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.015376 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.016366 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.016648 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.016704 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:38.018494 (+ 1790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:38.018495 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:38.018496 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:38.018502 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:38.022773 (+ 4271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:38.023155 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:38.023156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:38.023156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:38.023207 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:38.023394 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:38.023395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:38.023395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:38.023503 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:38.023593 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:38.024051 (+ 458us) server_negotiation.cc:299] Negotiation successful
0925 22:05:38.024132 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":48,"threads_started":1}
I20260925 22:05:38.024220 12454 heartbeater.cc:347] Connected to a master server at 127.5.0.124:40507
I20260925 22:05:38.024303 12454 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:38.024417 12456 heartbeater.cc:347] Connected to a master server at 127.5.0.125:40765
I20260925 22:05:38.024461 12456 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:38.024506 12454 heartbeater.cc:511] Master 127.5.0.124:40507 requested a full tablet report, sending...
I20260925 22:05:38.024528 12456 heartbeater.cc:511] Master 127.5.0.125:40765 requested a full tablet report, sending...
I20260925 22:05:38.025045 11816 ts_manager.cc:194] Registered new tserver with Master: 35c4eb82e71d482aac5e40b60a85dcbc (127.5.0.67:32981)
I20260925 22:05:38.025156 11993 ts_manager.cc:194] Registered new tserver with Master: 35c4eb82e71d482aac5e40b60a85dcbc (127.5.0.67:32981)
I20260925 22:05:38.025367 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:38.011381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:42711 (local address 127.5.0.126:41553)
0925 22:05:38.011427 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:38.011430 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:38.011651 (+ 221us) server_negotiation.cc:407] Connection header received
0925 22:05:38.012350 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:38.012352 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:38.012365 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:38.012398 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:38.014309 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.015027 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.018598 (+ 3571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.018874 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.018907 (+ 33us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:38.021252 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:38.021253 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:38.021254 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:38.021261 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:38.023001 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:38.023344 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:38.023345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:38.023345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:38.023409 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:38.024847 (+ 1438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:38.024848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:38.024849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:38.024970 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:38.025017 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:38.025212 (+ 195us) server_negotiation.cc:299] Negotiation successful
0925 22:05:38.025285 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 22:05:38.025728 12458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:38.011140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.5.0.67:42711)
0925 22:05:38.011599 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:38.011608 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:38.012225 (+ 617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:38.012913 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:38.012914 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:38.013207 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:38.013662 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:38.013668 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.015121 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.015122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:38.018497 (+ 3375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:38.018499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.019002 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.019002 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:38.019063 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:38.019655 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:38.019660 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:38.021147 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:38.023076 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:38.023076 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:38.023077 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:38.023265 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:38.023588 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:38.023588 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:38.023589 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:38.024764 (+ 1175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:38.025044 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:38.025045 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:38.025082 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 22:05:38.025513 (+ 431us) client_negotiation.cc:240] Negotiation successful
0925 22:05:38.025619 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"mutex_wait_us":2801,"thread_start_us":62,"threads_started":1}
I20260925 22:05:38.026149 12455 heartbeater.cc:347] Connected to a master server at 127.5.0.126:41553
I20260925 22:05:38.026214 12455 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:38.026283 12455 heartbeater.cc:511] Master 127.5.0.126:41553 requested a full tablet report, sending...
I20260925 22:05:38.026520 11215 ts_manager.cc:194] Re-registered known tserver with Master: 35c4eb82e71d482aac5e40b60a85dcbc (127.5.0.67:32981)
I20260925 22:05:38.026961 11215 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.0.67@KRBTEST.COM'} at 127.5.0.67:42711
I20260925 22:05:38.029419 12466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:38.012651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41802 (local address 127.5.0.126:41553)
0925 22:05:38.012911 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:38.012913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:38.013656 (+ 743us) server_negotiation.cc:407] Connection header received
0925 22:05:38.013680 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:38.013680 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:38.013693 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:38.013786 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:38.016083 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.016787 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.022113 (+ 5326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.022432 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.022494 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:38.026758 (+ 4264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:38.026759 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:38.026760 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:38.026766 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:38.028254 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:38.028601 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:38.028602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:38.028602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:38.028657 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:38.028855 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:38.028856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:38.028856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:38.029005 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:38.029111 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:38.029217 (+ 106us) server_negotiation.cc:299] Negotiation successful
0925 22:05:38.029284 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":69,"threads_started":1}
I20260925 22:05:38.030158 12469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:38.013221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59278 (local address 127.5.0.125:40765)
0925 22:05:38.013473 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:38.013475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:38.014060 (+ 585us) server_negotiation.cc:407] Connection header received
0925 22:05:38.014082 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:38.014083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:38.014094 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:38.014146 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:38.017514 (+ 3368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.018286 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.021589 (+ 3303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.021902 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.021967 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:38.026652 (+ 4685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:38.026653 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:38.026654 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:38.026660 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:38.028873 (+ 2213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:38.029202 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:38.029203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:38.029203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:38.029261 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:38.029629 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:38.029630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:38.029630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:38.029762 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:38.029839 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:38.029977 (+ 138us) server_negotiation.cc:299] Negotiation successful
0925 22:05:38.030062 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":48,"threads_started":1}
I20260925 22:05:38.030918 11215 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41802:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:05:38.032497 11215 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:05:38.044776 12476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:38.040127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:44879 (local address 127.0.0.1:35726)
0925 22:05:38.040514 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:38.040522 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:38.040555 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:38.040901 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:38.040902 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:38.040904 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:38.040969 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:38.040971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.042245 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.042247 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:38.043512 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:38.043516 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.044474 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.044475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:38.044573 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:38.044580 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 22:05:38.044635 (+ 55us) client_negotiation.cc:240] Negotiation successful
0925 22:05:38.044679 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":2,"thread_start_us":112,"threads_started":1}
I20260925 22:05:38.044791 12478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:38.040502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35726 (local address 127.5.0.65:44879)
0925 22:05:38.040674 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:38.040680 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:38.040690 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:05:38.040735 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:38.040736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:38.040753 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:38.040836 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:38.041090 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.042144 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.043594 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.044369 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.044476 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:38.044595 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:38.044653 (+ 58us) server_negotiation.cc:299] Negotiation successful
0925 22:05:38.044697 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":85,"threads_started":1}
I20260925 22:05:38.045013 12475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:38.040038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:34921 (local address 127.0.0.1:47132)
0925 22:05:38.040239 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:38.040280 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:38.040328 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:38.040856 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:38.040858 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:38.040861 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:38.040920 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:38.040923 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.042524 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.042525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:38.043650 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:38.043652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.044760 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.044761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:38.044826 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:38.044831 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:05:38.044887 (+ 56us) client_negotiation.cc:240] Negotiation successful
0925 22:05:38.044933 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174}
I20260925 22:05:38.045614 12477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:38.040284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47132 (local address 127.5.0.66:34921)
0925 22:05:38.040677 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:38.040681 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:38.040690 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:05:38.040734 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:38.040736 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:38.040753 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:38.040837 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:38.041014 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.042423 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.043884 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.044668 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.045397 (+ 729us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:38.045476 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:38.045500 (+ 24us) server_negotiation.cc:299] Negotiation successful
0925 22:05:38.045538 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":78,"threads_started":1}
I20260925 22:05:38.045673 11775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:38.040078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:32981 (local address 127.0.0.1:45834)
0925 22:05:38.040275 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:38.040286 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:38.040328 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:38.042335 (+ 2007us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:38.042336 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:38.042338 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:38.042368 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:38.042369 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.043615 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.043616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:38.044568 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:38.044570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.045446 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.045447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:38.045502 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:38.045506 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:05:38.045569 (+ 63us) client_negotiation.cc:240] Negotiation successful
0925 22:05:38.045608 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":3,"thread_start_us":76,"threads_started":1}
I20260925 22:05:38.045775 12115 tablet_service.cc:1467] Processing CreateTablet for tablet 8d754426611c4824b88830517971f5b4 (DEFAULT_TABLE table=test-table [id=039170b1f724412ab23d45a298aedbe7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:38.046093 12115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d754426611c4824b88830517971f5b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:38.046145 12252 tablet_service.cc:1467] Processing CreateTablet for tablet 8d754426611c4824b88830517971f5b4 (DEFAULT_TABLE table=test-table [id=039170b1f724412ab23d45a298aedbe7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:38.046406 12252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d754426611c4824b88830517971f5b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:38.048100 12479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:38.041815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45834 (local address 127.5.0.67:32981)
0925 22:05:38.042150 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:38.042155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:38.042165 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:05:38.042217 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:38.042219 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:38.042237 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:38.042432 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:38.042494 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.043518 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.044668 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.045352 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.045444 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:38.045519 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:38.045582 (+ 63us) server_negotiation.cc:299] Negotiation successful
0925 22:05:38.045618 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":80,"threads_started":1}
I20260925 22:05:38.048753 12388 tablet_service.cc:1467] Processing CreateTablet for tablet 8d754426611c4824b88830517971f5b4 (DEFAULT_TABLE table=test-table [id=039170b1f724412ab23d45a298aedbe7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:38.049110 12388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d754426611c4824b88830517971f5b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:38.049470 12481 tablet_bootstrap.cc:492] T 8d754426611c4824b88830517971f5b4 P e74b9533a2084cb5a37834fc583b7beb: Bootstrap starting.
I20260925 22:05:38.049659 12480 tablet_bootstrap.cc:492] T 8d754426611c4824b88830517971f5b4 P 2d23d1eb9a374b9a87383727acd62f84: Bootstrap starting.
I20260925 22:05:38.050098 12481 tablet_bootstrap.cc:654] T 8d754426611c4824b88830517971f5b4 P e74b9533a2084cb5a37834fc583b7beb: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:38.050604 12481 log.cc:824] T 8d754426611c4824b88830517971f5b4 P e74b9533a2084cb5a37834fc583b7beb: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:38.050743 12480 tablet_bootstrap.cc:654] T 8d754426611c4824b88830517971f5b4 P 2d23d1eb9a374b9a87383727acd62f84: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:38.051060 12480 log.cc:824] T 8d754426611c4824b88830517971f5b4 P 2d23d1eb9a374b9a87383727acd62f84: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:38.051826 12481 tablet_bootstrap.cc:492] T 8d754426611c4824b88830517971f5b4 P e74b9533a2084cb5a37834fc583b7beb: No bootstrap required, opened a new log
I20260925 22:05:38.051826 12480 tablet_bootstrap.cc:492] T 8d754426611c4824b88830517971f5b4 P 2d23d1eb9a374b9a87383727acd62f84: No bootstrap required, opened a new log
I20260925 22:05:38.051926 12481 ts_tablet_manager.cc:1397] T 8d754426611c4824b88830517971f5b4 P e74b9533a2084cb5a37834fc583b7beb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 22:05:38.051925 12480 ts_tablet_manager.cc:1397] T 8d754426611c4824b88830517971f5b4 P 2d23d1eb9a374b9a87383727acd62f84: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:05:38.052820 12484 tablet_bootstrap.cc:492] T 8d754426611c4824b88830517971f5b4 P 35c4eb82e71d482aac5e40b60a85dcbc: Bootstrap starting.
I20260925 22:05:38.053370 12484 tablet_bootstrap.cc:654] T 8d754426611c4824b88830517971f5b4 P 35c4eb82e71d482aac5e40b60a85dcbc: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:38.053567 12484 log.cc:824] T 8d754426611c4824b88830517971f5b4 P 35c4eb82e71d482aac5e40b60a85dcbc: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:38.053946 12481 raft_consensus.cc:348] T 8d754426611c4824b88830517971f5b4 P e74b9533a2084cb5a37834fc583b7beb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e74b9533a2084cb5a37834fc583b7beb" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 34921 } } peers { permanent_uuid: "2d23d1eb9a374b9a87383727acd62f84" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 44879 } } peers { permanent_uuid: "35c4eb82e71d482aac5e40b60a85dcbc" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32981 } }
I20260925 22:05:38.054116 12481 raft_consensus.cc:374] T 8d754426611c4824b88830517971f5b4 P e74b9533a2084cb5a37834fc583b7beb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:38.054162 12484 tablet_bootstrap.cc:492] T 8d754426611c4824b88830517971f5b4 P 35c4eb82e71d482aac5e40b60a85dcbc: No bootstrap required, opened a new log
I20260925 22:05:38.054152 12481 raft_consensus.cc:729] T 8d754426611c4824b88830517971f5b4 P e74b9533a2084cb5a37834fc583b7beb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e74b9533a2084cb5a37834fc583b7beb, State: Initialized, Role: FOLLOWER
I20260925 22:05:38.054210 12484 ts_tablet_manager.cc:1397] T 8d754426611c4824b88830517971f5b4 P 35c4eb82e71d482aac5e40b60a85dcbc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 22:05:38.054270 12481 consensus_queue.cc:260] T 8d754426611c4824b88830517971f5b4 P e74b9533a2084cb5a37834fc583b7beb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e74b9533a2084cb5a37834fc583b7beb" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 34921 } } peers { permanent_uuid: "2d23d1eb9a374b9a87383727acd62f84" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 44879 } } peers { permanent_uuid: "35c4eb82e71d482aac5e40b60a85dcbc" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32981 } }
I20260925 22:05:38.054528 12481 ts_tablet_manager.cc:1428] T 8d754426611c4824b88830517971f5b4 P e74b9533a2084cb5a37834fc583b7beb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 22:05:38.054445 12480 raft_consensus.cc:348] T 8d754426611c4824b88830517971f5b4 P 2d23d1eb9a374b9a87383727acd62f84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e74b9533a2084cb5a37834fc583b7beb" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 34921 } } peers { permanent_uuid: "2d23d1eb9a374b9a87383727acd62f84" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 44879 } } peers { permanent_uuid: "35c4eb82e71d482aac5e40b60a85dcbc" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32981 } }
I20260925 22:05:38.054594 12480 raft_consensus.cc:374] T 8d754426611c4824b88830517971f5b4 P 2d23d1eb9a374b9a87383727acd62f84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:38.054612 12319 heartbeater.cc:503] Master 127.5.0.126:41553 was elected leader, sending a full tablet report...
I20260925 22:05:38.054627 12480 raft_consensus.cc:729] T 8d754426611c4824b88830517971f5b4 P 2d23d1eb9a374b9a87383727acd62f84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d23d1eb9a374b9a87383727acd62f84, State: Initialized, Role: FOLLOWER
I20260925 22:05:38.054723 12480 consensus_queue.cc:260] T 8d754426611c4824b88830517971f5b4 P 2d23d1eb9a374b9a87383727acd62f84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e74b9533a2084cb5a37834fc583b7beb" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 34921 } } peers { permanent_uuid: "2d23d1eb9a374b9a87383727acd62f84" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 44879 } } peers { permanent_uuid: "35c4eb82e71d482aac5e40b60a85dcbc" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32981 } }
I20260925 22:05:38.054960 12480 ts_tablet_manager.cc:1428] T 8d754426611c4824b88830517971f5b4 P 2d23d1eb9a374b9a87383727acd62f84: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 22:05:38.055531 12182 heartbeater.cc:503] Master 127.5.0.126:41553 was elected leader, sending a full tablet report...
I20260925 22:05:38.057394 12484 raft_consensus.cc:348] T 8d754426611c4824b88830517971f5b4 P 35c4eb82e71d482aac5e40b60a85dcbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e74b9533a2084cb5a37834fc583b7beb" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 34921 } } peers { permanent_uuid: "2d23d1eb9a374b9a87383727acd62f84" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 44879 } } peers { permanent_uuid: "35c4eb82e71d482aac5e40b60a85dcbc" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32981 } }
I20260925 22:05:38.057561 12484 raft_consensus.cc:374] T 8d754426611c4824b88830517971f5b4 P 35c4eb82e71d482aac5e40b60a85dcbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:38.057618 12484 raft_consensus.cc:729] T 8d754426611c4824b88830517971f5b4 P 35c4eb82e71d482aac5e40b60a85dcbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35c4eb82e71d482aac5e40b60a85dcbc, State: Initialized, Role: FOLLOWER
I20260925 22:05:38.057760 12484 consensus_queue.cc:260] T 8d754426611c4824b88830517971f5b4 P 35c4eb82e71d482aac5e40b60a85dcbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e74b9533a2084cb5a37834fc583b7beb" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 34921 } } peers { permanent_uuid: "2d23d1eb9a374b9a87383727acd62f84" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 44879 } } peers { permanent_uuid: "35c4eb82e71d482aac5e40b60a85dcbc" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32981 } }
I20260925 22:05:38.058060 12455 heartbeater.cc:503] Master 127.5.0.126:41553 was elected leader, sending a full tablet report...
I20260925 22:05:38.058218 12484 ts_tablet_manager.cc:1428] T 8d754426611c4824b88830517971f5b4 P 35c4eb82e71d482aac5e40b60a85dcbc: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260925 22:05:38.102849 11775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:38.100462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:40507 (local address 127.0.0.1:45540)
0925 22:05:38.100522 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:38.100537 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:38.100572 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:38.100754 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:38.100755 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:38.100756 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:38.100805 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:38.100807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.101403 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.101404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:38.102136 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:38.102138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.102669 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.102678 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:38.102721 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:38.102724 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:05:38.102762 (+ 38us) client_negotiation.cc:240] Negotiation successful
0925 22:05:38.102794 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260925 22:05:38.102846 12468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:38.100555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45540 (local address 127.5.0.124:40507)
0925 22:05:38.100621 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:38.100623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:38.100631 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:05:38.100653 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:38.100654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:38.100672 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:38.100745 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:38.100866 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.101348 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.102215 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.102600 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.102660 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:38.102721 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:38.102765 (+ 44us) server_negotiation.cc:299] Negotiation successful
0925 22:05:38.102792 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 22:05:38.103235 12010 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260925 22:05:38.104051 12041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8faad6f3402b4d1794676034a93d324a. Latest consensus state: current_term: 1 leader_uuid: "8faad6f3402b4d1794676034a93d324a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: false } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: NON_VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: true } } }
I20260925 22:05:38.104156 12041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [sys.catalog]: This master's current role is: LEARNER
W20260925 22:05:38.114555 12321 tablet.cc:2369] T 8d754426611c4824b88830517971f5b4 P e74b9533a2084cb5a37834fc583b7beb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 22:05:38.220600 12184 tablet.cc:2369] T 8d754426611c4824b88830517971f5b4 P 2d23d1eb9a374b9a87383727acd62f84: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:05:38.234491 12487 raft_consensus.cc:482] T 8d754426611c4824b88830517971f5b4 P 2d23d1eb9a374b9a87383727acd62f84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:05:38.234751 12487 raft_consensus.cc:504] T 8d754426611c4824b88830517971f5b4 P 2d23d1eb9a374b9a87383727acd62f84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e74b9533a2084cb5a37834fc583b7beb" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 34921 } } peers { permanent_uuid: "2d23d1eb9a374b9a87383727acd62f84" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 44879 } } peers { permanent_uuid: "35c4eb82e71d482aac5e40b60a85dcbc" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32981 } }
I20260925 22:05:38.235131 12487 leader_election.cc:288] T 8d754426611c4824b88830517971f5b4 P 2d23d1eb9a374b9a87383727acd62f84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e74b9533a2084cb5a37834fc583b7beb (127.5.0.66:34921), 35c4eb82e71d482aac5e40b60a85dcbc (127.5.0.67:32981)
I20260925 22:05:38.237989 12477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:38.235412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:56311 (local address 127.5.0.66:34921)
0925 22:05:38.235461 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:38.235463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:38.235571 (+ 108us) server_negotiation.cc:407] Connection header received
0925 22:05:38.235640 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:38.235642 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:38.235650 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:38.235714 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:38.235842 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.236328 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.237285 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.237646 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.237731 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:38.237741 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:38.237880 (+ 139us) server_negotiation.cc:299] Negotiation successful
0925 22:05:38.237915 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 22:05:38.238004 12479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:38.235407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:53351 (local address 127.5.0.67:32981)
0925 22:05:38.235469 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:38.235471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:38.235502 (+ 31us) server_negotiation.cc:407] Connection header received
0925 22:05:38.235530 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:38.235531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:38.235546 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:38.235613 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:38.235763 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.236298 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.237287 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.237651 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.237737 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:38.237746 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:38.237888 (+ 142us) server_negotiation.cc:299] Negotiation successful
0925 22:05:38.237925 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 22:05:38.238013 12185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:38.235318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:32981 (local address 127.5.0.65:53351)
0925 22:05:38.235372 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:38.235387 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:38.235453 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:38.235619 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:38.235619 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:38.235621 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:38.235692 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:38.235695 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.236388 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.236389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:38.237190 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:38.237193 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.237739 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.237740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:38.237809 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:38.237812 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:05:38.237880 (+ 68us) client_negotiation.cc:240] Negotiation successful
0925 22:05:38.237929 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":18}
I20260925 22:05:38.238014 12493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:38.235318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:34921 (local address 127.5.0.65:56311)
0925 22:05:38.235508 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:38.235516 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:38.235586 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:38.235717 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:38.235718 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:38.235719 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:38.235779 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:38.235780 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.236407 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.236408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:38.237200 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:38.237201 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.237732 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.237733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:38.237802 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:38.237805 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:05:38.237873 (+ 68us) client_negotiation.cc:240] Negotiation successful
0925 22:05:38.237915 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":100,"thread_start_us":62,"threads_started":1}
I20260925 22:05:38.238411 12272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d754426611c4824b88830517971f5b4" candidate_uuid: "2d23d1eb9a374b9a87383727acd62f84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e74b9533a2084cb5a37834fc583b7beb" is_pre_election: true
I20260925 22:05:38.238474 12408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d754426611c4824b88830517971f5b4" candidate_uuid: "2d23d1eb9a374b9a87383727acd62f84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35c4eb82e71d482aac5e40b60a85dcbc" is_pre_election: true
I20260925 22:05:38.238616 12272 raft_consensus.cc:2454] T 8d754426611c4824b88830517971f5b4 P e74b9533a2084cb5a37834fc583b7beb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d23d1eb9a374b9a87383727acd62f84 in term 0.
I20260925 22:05:38.238631 12408 raft_consensus.cc:2454] T 8d754426611c4824b88830517971f5b4 P 35c4eb82e71d482aac5e40b60a85dcbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d23d1eb9a374b9a87383727acd62f84 in term 0.
I20260925 22:05:38.238847 12071 leader_election.cc:302] T 8d754426611c4824b88830517971f5b4 P 2d23d1eb9a374b9a87383727acd62f84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d23d1eb9a374b9a87383727acd62f84, 35c4eb82e71d482aac5e40b60a85dcbc; no voters:
I20260925 22:05:38.239009 12487 raft_consensus.cc:2781] T 8d754426611c4824b88830517971f5b4 P 2d23d1eb9a374b9a87383727acd62f84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:05:38.239084 12487 raft_consensus.cc:482] T 8d754426611c4824b88830517971f5b4 P 2d23d1eb9a374b9a87383727acd62f84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:05:38.239111 12487 raft_consensus.cc:3037] T 8d754426611c4824b88830517971f5b4 P 2d23d1eb9a374b9a87383727acd62f84 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:38.239789 12487 raft_consensus.cc:504] T 8d754426611c4824b88830517971f5b4 P 2d23d1eb9a374b9a87383727acd62f84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e74b9533a2084cb5a37834fc583b7beb" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 34921 } } peers { permanent_uuid: "2d23d1eb9a374b9a87383727acd62f84" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 44879 } } peers { permanent_uuid: "35c4eb82e71d482aac5e40b60a85dcbc" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32981 } }
I20260925 22:05:38.239955 12487 leader_election.cc:288] T 8d754426611c4824b88830517971f5b4 P 2d23d1eb9a374b9a87383727acd62f84 [CANDIDATE]: Term 1 election: Requested vote from peers e74b9533a2084cb5a37834fc583b7beb (127.5.0.66:34921), 35c4eb82e71d482aac5e40b60a85dcbc (127.5.0.67:32981)
I20260925 22:05:38.240074 12272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d754426611c4824b88830517971f5b4" candidate_uuid: "2d23d1eb9a374b9a87383727acd62f84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e74b9533a2084cb5a37834fc583b7beb"
I20260925 22:05:38.240108 12408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d754426611c4824b88830517971f5b4" candidate_uuid: "2d23d1eb9a374b9a87383727acd62f84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35c4eb82e71d482aac5e40b60a85dcbc"
I20260925 22:05:38.240171 12272 raft_consensus.cc:3037] T 8d754426611c4824b88830517971f5b4 P e74b9533a2084cb5a37834fc583b7beb [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:38.240186 12408 raft_consensus.cc:3037] T 8d754426611c4824b88830517971f5b4 P 35c4eb82e71d482aac5e40b60a85dcbc [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:38.241040 12408 raft_consensus.cc:2454] T 8d754426611c4824b88830517971f5b4 P 35c4eb82e71d482aac5e40b60a85dcbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d23d1eb9a374b9a87383727acd62f84 in term 1.
I20260925 22:05:38.241040 12272 raft_consensus.cc:2454] T 8d754426611c4824b88830517971f5b4 P e74b9533a2084cb5a37834fc583b7beb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d23d1eb9a374b9a87383727acd62f84 in term 1.
I20260925 22:05:38.241216 12071 leader_election.cc:302] T 8d754426611c4824b88830517971f5b4 P 2d23d1eb9a374b9a87383727acd62f84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d23d1eb9a374b9a87383727acd62f84, 35c4eb82e71d482aac5e40b60a85dcbc; no voters:
I20260925 22:05:38.241334 12487 raft_consensus.cc:2781] T 8d754426611c4824b88830517971f5b4 P 2d23d1eb9a374b9a87383727acd62f84 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:38.241457 12487 raft_consensus.cc:686] T 8d754426611c4824b88830517971f5b4 P 2d23d1eb9a374b9a87383727acd62f84 [term 1 LEADER]: Becoming Leader. State: Replica: 2d23d1eb9a374b9a87383727acd62f84, State: Running, Role: LEADER
I20260925 22:05:38.241571 12487 consensus_queue.cc:237] T 8d754426611c4824b88830517971f5b4 P 2d23d1eb9a374b9a87383727acd62f84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e74b9533a2084cb5a37834fc583b7beb" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 34921 } } peers { permanent_uuid: "2d23d1eb9a374b9a87383727acd62f84" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 44879 } } peers { permanent_uuid: "35c4eb82e71d482aac5e40b60a85dcbc" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32981 } }
I20260925 22:05:38.242381 11215 catalog_manager.cc:5495] T 8d754426611c4824b88830517971f5b4 P 2d23d1eb9a374b9a87383727acd62f84 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d23d1eb9a374b9a87383727acd62f84 (127.5.0.65). New cstate: current_term: 1 leader_uuid: "2d23d1eb9a374b9a87383727acd62f84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e74b9533a2084cb5a37834fc583b7beb" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 34921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d23d1eb9a374b9a87383727acd62f84" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 44879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35c4eb82e71d482aac5e40b60a85dcbc" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32981 } health_report { overall_health: UNKNOWN } } }
I20260925 22:05:38.243075 11966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8faad6f3402b4d1794676034a93d324a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: false } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: NON_VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: true } } }
I20260925 22:05:38.243171 11966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:38.243851 12496 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a: attempting to promote NON_VOTER 02ee5b2754924933997a4ba119c02639 to VOTER
I20260925 22:05:38.244005 12496 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: false } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: false } }
I20260925 22:05:38.244351 12010 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [term 1 LEARNER]: Refusing update from remote peer 8faad6f3402b4d1794676034a93d324a: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260925 22:05:38.244601 11825 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [term 1 FOLLOWER]: Refusing update from remote peer 8faad6f3402b4d1794676034a93d324a: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260925 22:05:38.244590 12497 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [LEADER]: Connected to new peer: Peer: permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260925 22:05:38.244786 12497 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260925 22:05:38.245400 12497 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 02ee5b2754924933997a4ba119c02639 (127.5.0.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: false } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: false } } }
I20260925 22:05:38.245581 11825 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 02ee5b2754924933997a4ba119c02639 (127.5.0.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: false } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: false } } }
I20260925 22:05:38.245630 12010 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 02ee5b2754924933997a4ba119c02639 (127.5.0.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: false } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: false } } }
I20260925 22:05:38.246292 12497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8faad6f3402b4d1794676034a93d324a" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: false } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: false } } }
I20260925 22:05:38.246369 12497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:38.246424 12497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8faad6f3402b4d1794676034a93d324a" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: false } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: false } } }
I20260925 22:05:38.246495 12497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:38.246479 12496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8faad6f3402b4d1794676034a93d324a" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: false } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: false } } }
I20260925 22:05:38.246635 12496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:38.246630 12041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8faad6f3402b4d1794676034a93d324a" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: false } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: false } } }
I20260925 22:05:38.246717 12041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:05:38.246753 12499 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8faad6f3402b4d1794676034a93d324a" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: false } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: false } } }
I20260925 22:05:38.246843 12499 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [sys.catalog]: This master's current role is: FOLLOWER
W20260925 22:05:38.261228 12457 tablet.cc:2369] T 8d754426611c4824b88830517971f5b4 P 35c4eb82e71d482aac5e40b60a85dcbc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:05:38.265933 12478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:38.263468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35732 (local address 127.5.0.65:44879)
0925 22:05:38.263532 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:38.263534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:38.263571 (+ 37us) server_negotiation.cc:407] Connection header received
0925 22:05:38.263615 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:38.263616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:38.263635 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:38.263712 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:05:38.263841 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.264370 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.265127 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.265319 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.265398 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:38.265566 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:05:38.265699 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:05:38.265791 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:38.265810 (+ 19us) server_negotiation.cc:299] Negotiation successful
0925 22:05:38.265831 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 22:05:38.267825 12408 raft_consensus.cc:1261] T 8d754426611c4824b88830517971f5b4 P 35c4eb82e71d482aac5e40b60a85dcbc [term 1 FOLLOWER]: Refusing update from remote peer 2d23d1eb9a374b9a87383727acd62f84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:05:38.267872 12272 raft_consensus.cc:1261] T 8d754426611c4824b88830517971f5b4 P e74b9533a2084cb5a37834fc583b7beb [term 1 FOLLOWER]: Refusing update from remote peer 2d23d1eb9a374b9a87383727acd62f84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:05:38.268244 12487 consensus_queue.cc:1035] T 8d754426611c4824b88830517971f5b4 P 2d23d1eb9a374b9a87383727acd62f84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35c4eb82e71d482aac5e40b60a85dcbc" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:05:38.268428 12494 consensus_queue.cc:1035] T 8d754426611c4824b88830517971f5b4 P 2d23d1eb9a374b9a87383727acd62f84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e74b9533a2084cb5a37834fc583b7beb" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 34921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:05:38.272425 12503 mvcc.cc:205] Tried to move back new op lower bound from 7333371651142381568 to 7333371651038126080. Current Snapshot: MvccSnapshot[applied={T|T < 7333371651142381568 or (T in {7333371651142381568})}]
I20260925 22:05:38.273484 12502 mvcc.cc:205] Tried to move back new op lower bound from 7333371651142381568 to 7333371651038126080. Current Snapshot: MvccSnapshot[applied={T|T < 7333371651142381568 or (T in {7333371651142381568})}]
I20260925 22:05:38.274222 11214 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:05:38.274328 11214 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:05:38.275977 11214 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8faad6f3402b4d1794676034a93d324a: Sending DeleteTablet for 3 replicas of tablet 8d754426611c4824b88830517971f5b4
I20260925 22:05:38.276285 12252 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d754426611c4824b88830517971f5b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:38 UTC) from {username='oryx', principal='oryx/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:47132
I20260925 22:05:38.276309 12115 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d754426611c4824b88830517971f5b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:38 UTC) from {username='oryx', principal='oryx/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:35726
I20260925 22:05:38.276382 12388 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d754426611c4824b88830517971f5b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:38 UTC) from {username='oryx', principal='oryx/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:45834
I20260925 22:05:38.277316 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 11183
I20260925 22:05:38.277339 12511 tablet_replica.cc:330] T 8d754426611c4824b88830517971f5b4 P 2d23d1eb9a374b9a87383727acd62f84: stopping tablet replica
I20260925 22:05:38.278167 12511 raft_consensus.cc:2229] T 8d754426611c4824b88830517971f5b4 P 2d23d1eb9a374b9a87383727acd62f84 [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:05:38.278328 12511 raft_consensus.cc:2258] T 8d754426611c4824b88830517971f5b4 P 2d23d1eb9a374b9a87383727acd62f84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:38.278784 12511 ts_tablet_manager.cc:1905] T 8d754426611c4824b88830517971f5b4 P 2d23d1eb9a374b9a87383727acd62f84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:38.280499 12511 ts_tablet_manager.cc:1918] T 8d754426611c4824b88830517971f5b4 P 2d23d1eb9a374b9a87383727acd62f84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:05:38.280573 12511 log.cc:1196] T 8d754426611c4824b88830517971f5b4 P 2d23d1eb9a374b9a87383727acd62f84: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-0/wal/wals/8d754426611c4824b88830517971f5b4
I20260925 22:05:38.280838 12511 ts_tablet_manager.cc:1939] T 8d754426611c4824b88830517971f5b4 P 2d23d1eb9a374b9a87383727acd62f84: Deleting consensus metadata
I20260925 22:05:38.281301 12512 tablet_replica.cc:330] T 8d754426611c4824b88830517971f5b4 P e74b9533a2084cb5a37834fc583b7beb: stopping tablet replica
I20260925 22:05:38.281424 12512 raft_consensus.cc:2229] T 8d754426611c4824b88830517971f5b4 P e74b9533a2084cb5a37834fc583b7beb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:05:38.281529 12512 raft_consensus.cc:2258] T 8d754426611c4824b88830517971f5b4 P e74b9533a2084cb5a37834fc583b7beb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:38.282086 12512 ts_tablet_manager.cc:1905] T 8d754426611c4824b88830517971f5b4 P e74b9533a2084cb5a37834fc583b7beb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 22:05:38.283671 12071 connection.cc:535] server connection from 127.0.0.1:35726 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 22:05:38.283975 12512 ts_tablet_manager.cc:1918] T 8d754426611c4824b88830517971f5b4 P e74b9533a2084cb5a37834fc583b7beb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:05:38.284325 12512 log.cc:1196] T 8d754426611c4824b88830517971f5b4 P e74b9533a2084cb5a37834fc583b7beb: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-1/wal/wals/8d754426611c4824b88830517971f5b4
I20260925 22:05:38.284793 12512 ts_tablet_manager.cc:1939] T 8d754426611c4824b88830517971f5b4 P e74b9533a2084cb5a37834fc583b7beb: Deleting consensus metadata
W20260925 22:05:38.285213 12207 connection.cc:441] server connection from 127.0.0.1:47132 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47132 (request call id 1) could send its response
I20260925 22:05:38.286250 12468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:38.276731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45556 (local address 127.5.0.124:40507)
0925 22:05:38.276777 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:38.276779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:38.276904 (+ 125us) server_negotiation.cc:407] Connection header received
0925 22:05:38.277002 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:38.277003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:38.277014 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:38.277042 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:38.278436 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.279242 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.280500 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.280848 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.280894 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:38.283052 (+ 2158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:38.283053 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:38.283054 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:38.283062 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:38.284936 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:38.285370 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:38.285372 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:38.285372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:38.285433 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:38.285658 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:38.285659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:38.285659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:38.285814 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:38.285934 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:38.286059 (+ 125us) server_negotiation.cc:299] Negotiation successful
0925 22:05:38.286146 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 22:05:38.288087 12513 tablet_replica.cc:330] T 8d754426611c4824b88830517971f5b4 P 35c4eb82e71d482aac5e40b60a85dcbc: stopping tablet replica
I20260925 22:05:38.288228 12513 raft_consensus.cc:2229] T 8d754426611c4824b88830517971f5b4 P 35c4eb82e71d482aac5e40b60a85dcbc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:05:38.288357 12513 raft_consensus.cc:2258] T 8d754426611c4824b88830517971f5b4 P 35c4eb82e71d482aac5e40b60a85dcbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:38.288946 12513 ts_tablet_manager.cc:1905] T 8d754426611c4824b88830517971f5b4 P 35c4eb82e71d482aac5e40b60a85dcbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:38.290709 12513 ts_tablet_manager.cc:1918] T 8d754426611c4824b88830517971f5b4 P 35c4eb82e71d482aac5e40b60a85dcbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:05:38.290783 12513 log.cc:1196] T 8d754426611c4824b88830517971f5b4 P 35c4eb82e71d482aac5e40b60a85dcbc: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-2/wal/wals/8d754426611c4824b88830517971f5b4
I20260925 22:05:38.291076 12513 ts_tablet_manager.cc:1939] T 8d754426611c4824b88830517971f5b4 P 35c4eb82e71d482aac5e40b60a85dcbc: Deleting consensus metadata
W20260925 22:05:38.291496 12343 connection.cc:441] server connection from 127.0.0.1:45834 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45834 (request call id 1) could send its response
I20260925 22:05:38.296918 12469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:38.286271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59290 (local address 127.5.0.125:40765)
0925 22:05:38.286317 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:38.286319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:38.286499 (+ 180us) server_negotiation.cc:407] Connection header received
0925 22:05:38.286672 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:38.286674 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:38.286688 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:38.286740 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:38.287325 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.287846 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.291607 (+ 3761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.291956 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.292000 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:38.293704 (+ 1704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:38.293706 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:38.293706 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:38.293715 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:38.295623 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:38.296022 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:38.296023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:38.296023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:38.296083 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:38.296288 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:38.296288 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:38.296289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:38.296422 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:38.296522 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:38.296703 (+ 181us) server_negotiation.cc:299] Negotiation successful
0925 22:05:38.296790 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 22:05:38.296932 12468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:38.288650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45560 (local address 127.5.0.124:40507)
0925 22:05:38.288691 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:38.288693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:38.288815 (+ 122us) server_negotiation.cc:407] Connection header received
0925 22:05:38.288906 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:38.288907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:38.288917 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:38.288944 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:38.290054 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.290761 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.292003 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:38.292318 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:38.292359 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:38.293731 (+ 1372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:38.293732 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:38.293733 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:38.293739 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:38.295373 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:38.295754 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:38.295756 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:38.295756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:38.295808 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:38.296081 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:38.296082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:38.296083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:38.296222 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:38.296324 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:38.296730 (+ 406us) server_negotiation.cc:299] Negotiation successful
0925 22:05:38.296836 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 22:05:39.272943 12522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:39.272680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.5.0.66:53405)
0925 22:05:39.272821 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:39.272894 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":63,"threads_started":1}
I20260925 22:05:39.272923 12521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:39.272590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.5.0.65:50673)
0925 22:05:39.272774 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:39.272859 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":66,"threads_started":1}
I20260925 22:05:39.273180 12523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:39.272925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.5.0.67:51645)
0925 22:05:39.273072 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:39.273133 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":47,"threads_started":1}
W20260925 22:05:39.273276 12182 heartbeater.cc:644] Failed to heartbeat to 127.5.0.126:41553 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
W20260925 22:05:39.273391 12319 heartbeater.cc:644] Failed to heartbeat to 127.5.0.126:41553 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
W20260925 22:05:39.273617 12455 heartbeater.cc:644] Failed to heartbeat to 127.5.0.126:41553 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
I20260925 22:05:39.776360 12524 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8faad6f3402b4d1794676034a93d324a)
I20260925 22:05:39.776484 12524 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: false } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: false } }
I20260925 22:05:39.776738 12524 leader_election.cc:288] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8faad6f3402b4d1794676034a93d324a (127.5.0.126:41553), 02ee5b2754924933997a4ba119c02639 (127.5.0.124:40507)
I20260925 22:05:39.776837 12525 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8faad6f3402b4d1794676034a93d324a)
I20260925 22:05:39.776959 12525 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: false } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: false } }
I20260925 22:05:39.777066 12527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:39.776840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.0.0.1:42294)
0925 22:05:39.776957 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:39.777012 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":44,"threads_started":1}
W20260925 22:05:39.777223 11802 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
I20260925 22:05:39.777472 12529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:39.777264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.0.0.1:42310)
0925 22:05:39.777350 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:39.777426 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70,"thread_start_us":44,"threads_started":1}
I20260925 22:05:39.777680 12527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:39.777532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.0.0.1:42322)
0925 22:05:39.777622 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:39.777649 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260925 22:05:39.777774 11802 leader_election.cc:334] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8faad6f3402b4d1794676034a93d324a (127.5.0.126:41553): Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
I20260925 22:05:39.777513 12525 leader_election.cc:288] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8faad6f3402b4d1794676034a93d324a (127.5.0.126:41553), f6d92dd7ba964ba6b2f2b1a84b443f9f (127.5.0.125:40765)
W20260925 22:05:39.778003 11988 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
I20260925 22:05:39.778425 12529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:39.778336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.0.0.1:42324)
0925 22:05:39.778375 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:39.778396 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260925 22:05:39.778502 11988 leader_election.cc:334] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8faad6f3402b4d1794676034a93d324a (127.5.0.126:41553): Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
I20260925 22:05:39.780530 12532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:39.777562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59306 (local address 127.5.0.125:40765)
0925 22:05:39.777746 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:39.777748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:39.777757 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:05:39.777777 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:39.777778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:39.777787 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:39.777849 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:39.778089 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:39.778816 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:39.779765 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:39.780211 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:39.780292 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:39.780303 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:39.780422 (+ 119us) server_negotiation.cc:299] Negotiation successful
0925 22:05:39.780462 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":84,"threads_started":1}
I20260925 22:05:39.780541 12531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:39.777434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:40765 (local address 127.0.0.1:59306)
0925 22:05:39.777527 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:39.777537 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:39.777586 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:39.777886 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:39.777887 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:39.777889 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:39.777946 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:39.777948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:39.778901 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:39.778902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:39.779670 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:39.779672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:39.780290 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:39.780292 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:39.780348 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:39.780350 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 22:05:39.780402 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 22:05:39.780462 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":51,"threads_started":1}
I20260925 22:05:39.780531 12526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:39.776857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:40507 (local address 127.0.0.1:45564)
0925 22:05:39.777034 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:39.777107 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:39.777398 (+ 291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:39.777856 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:39.777856 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:39.777858 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:39.777902 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:39.777904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:39.778922 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:39.778922 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:39.779669 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:39.779671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:39.780302 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:39.780303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:39.780363 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:39.780365 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 22:05:39.780435 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 22:05:39.780465 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":51,"threads_started":1}
I20260925 22:05:39.780922 11825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "02ee5b2754924933997a4ba119c02639" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" is_pre_election: true
I20260925 22:05:39.780543 12528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:39.776977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45564 (local address 127.5.0.124:40507)
0925 22:05:39.777079 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:39.777081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:39.777181 (+ 100us) server_negotiation.cc:407] Connection header received
0925 22:05:39.777726 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:39.777727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:39.777738 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:39.777800 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:39.778015 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:39.778836 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:39.779778 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:39.780210 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:39.780299 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:39.780308 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:39.780445 (+ 137us) server_negotiation.cc:299] Negotiation successful
0925 22:05:39.780479 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":42,"threads_started":1}
I20260925 22:05:39.781028 11825 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02ee5b2754924933997a4ba119c02639 in term 1.
I20260925 22:05:39.781152 12010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "02ee5b2754924933997a4ba119c02639" is_pre_election: true
I20260925 22:05:39.781255 12010 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6d92dd7ba964ba6b2f2b1a84b443f9f in term 1.
I20260925 22:05:39.781428 11987 leader_election.cc:302] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02ee5b2754924933997a4ba119c02639, f6d92dd7ba964ba6b2f2b1a84b443f9f; no voters: 8faad6f3402b4d1794676034a93d324a
I20260925 22:05:39.781452 11800 leader_election.cc:302] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02ee5b2754924933997a4ba119c02639, f6d92dd7ba964ba6b2f2b1a84b443f9f; no voters: 8faad6f3402b4d1794676034a93d324a
I20260925 22:05:39.781546 12525 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 22:05:39.781564 12524 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 22:05:39.781591 12525 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8faad6f3402b4d1794676034a93d324a)
I20260925 22:05:39.781603 12524 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8faad6f3402b4d1794676034a93d324a)
I20260925 22:05:39.781611 12525 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [term 1 FOLLOWER]: Advancing to term 2
I20260925 22:05:39.781639 12524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [term 1 FOLLOWER]: Advancing to term 2
I20260925 22:05:39.782294 12525 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: false } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: false } }
I20260925 22:05:39.782300 12524 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: false } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: false } }
I20260925 22:05:39.782441 12525 leader_election.cc:288] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [CANDIDATE]: Term 2 election: Requested vote from peers 8faad6f3402b4d1794676034a93d324a (127.5.0.126:41553), f6d92dd7ba964ba6b2f2b1a84b443f9f (127.5.0.125:40765)
I20260925 22:05:39.782446 12524 leader_election.cc:288] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [CANDIDATE]: Term 2 election: Requested vote from peers 8faad6f3402b4d1794676034a93d324a (127.5.0.126:41553), 02ee5b2754924933997a4ba119c02639 (127.5.0.124:40507)
I20260925 22:05:39.782605 12531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:39.782508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.0.0.1:42334)
0925 22:05:39.782545 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:39.782576 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260925 22:05:39.782653 12010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "02ee5b2754924933997a4ba119c02639"
I20260925 22:05:39.782743 12010 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f6d92dd7ba964ba6b2f2b1a84b443f9f in current term 2: Already voted for candidate 02ee5b2754924933997a4ba119c02639 in this term.
I20260925 22:05:39.782708 11825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "02ee5b2754924933997a4ba119c02639" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f"
I20260925 22:05:39.782761 12526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:39.782563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.0.0.1:42350)
0925 22:05:39.782612 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:39.782639 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260925 22:05:39.782790 11825 raft_consensus.cc:2379] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 02ee5b2754924933997a4ba119c02639 in current term 2: Already voted for candidate f6d92dd7ba964ba6b2f2b1a84b443f9f in this term.
I20260925 22:05:39.783038 12526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:39.782967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.0.0.1:42358)
0925 22:05:39.783003 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:39.783025 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260925 22:05:39.783136 12531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:39.783035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.0.0.1:42360)
0925 22:05:39.783090 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:39.783114 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260925 22:05:39.783178 11802 leader_election.cc:334] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8faad6f3402b4d1794676034a93d324a (127.5.0.126:41553): Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
W20260925 22:05:39.783201 11988 leader_election.cc:334] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8faad6f3402b4d1794676034a93d324a (127.5.0.126:41553): Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
I20260925 22:05:39.783224 11802 leader_election.cc:302] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6d92dd7ba964ba6b2f2b1a84b443f9f; no voters: 02ee5b2754924933997a4ba119c02639, 8faad6f3402b4d1794676034a93d324a
I20260925 22:05:39.783265 11988 leader_election.cc:302] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 02ee5b2754924933997a4ba119c02639; no voters: 8faad6f3402b4d1794676034a93d324a, f6d92dd7ba964ba6b2f2b1a84b443f9f
I20260925 22:05:39.783313 12524 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260925 22:05:39.783344 12525 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260925 22:05:40.273885 12535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:40.273654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.5.0.66:57011)
0925 22:05:40.273792 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:40.273829 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":80,"threads_started":1}
I20260925 22:05:40.273888 12534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:40.273615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.5.0.65:58069)
0925 22:05:40.273782 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:40.273829 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":113,"threads_started":1}
I20260925 22:05:40.274122 12536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:40.273911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.5.0.67:45189)
0925 22:05:40.274045 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:40.274081 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":58,"threads_started":1}
I20260925 22:05:41.274624 12537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:41.274298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.5.0.66:53721)
0925 22:05:41.274445 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:41.274489 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":87,"threads_started":1}
I20260925 22:05:41.274849 12539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:41.274637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.5.0.67:59143)
0925 22:05:41.274774 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:41.274807 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":44,"threads_started":1}
I20260925 22:05:41.274868 12538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:41.274400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.5.0.65:36841)
0925 22:05:41.274623 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:41.274658 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":56,"threads_started":1}
I20260925 22:05:41.365423 12540 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:05:41.365525 12540 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: false } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: false } }
I20260925 22:05:41.365715 12540 leader_election.cc:288] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8faad6f3402b4d1794676034a93d324a (127.5.0.126:41553), f6d92dd7ba964ba6b2f2b1a84b443f9f (127.5.0.125:40765)
I20260925 22:05:41.365972 11825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "02ee5b2754924933997a4ba119c02639" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" is_pre_election: true
I20260925 22:05:41.366087 12541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:41.365856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.0.0.1:42534)
0925 22:05:41.365994 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:41.366040 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":58,"threads_started":1}
I20260925 22:05:41.366091 11825 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02ee5b2754924933997a4ba119c02639 in term 2.
I20260925 22:05:41.366235 11987 leader_election.cc:302] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02ee5b2754924933997a4ba119c02639, f6d92dd7ba964ba6b2f2b1a84b443f9f; no voters:
I20260925 22:05:41.366338 12540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260925 22:05:41.366390 12540 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:05:41.366405 12540 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [term 2 FOLLOWER]: Advancing to term 3
I20260925 22:05:41.367105 12540 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: false } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: false } }
I20260925 22:05:41.367209 12540 leader_election.cc:288] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [CANDIDATE]: Term 3 election: Requested vote from peers 8faad6f3402b4d1794676034a93d324a (127.5.0.126:41553), f6d92dd7ba964ba6b2f2b1a84b443f9f (127.5.0.125:40765)
I20260925 22:05:41.367436 11825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "02ee5b2754924933997a4ba119c02639" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f"
I20260925 22:05:41.367522 11825 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [term 2 FOLLOWER]: Advancing to term 3
I20260925 22:05:41.367548 12541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:41.367362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.0.0.1:42550)
0925 22:05:41.367412 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:41.367449 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260925 22:05:41.367623 11988 leader_election.cc:334] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8faad6f3402b4d1794676034a93d324a (127.5.0.126:41553): Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
I20260925 22:05:41.368049 12541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:41.367951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.0.0.1:42566)
0925 22:05:41.368004 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:41.368030 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260925 22:05:41.368145 11988 leader_election.cc:334] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 8faad6f3402b4d1794676034a93d324a (127.5.0.126:41553): Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
I20260925 22:05:41.368487 11825 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02ee5b2754924933997a4ba119c02639 in term 3.
I20260925 22:05:41.368660 11987 leader_election.cc:302] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02ee5b2754924933997a4ba119c02639, f6d92dd7ba964ba6b2f2b1a84b443f9f; no voters: 8faad6f3402b4d1794676034a93d324a
I20260925 22:05:41.368752 12540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [term 3 FOLLOWER]: Leader election won for term 3
I20260925 22:05:41.368865 12540 raft_consensus.cc:686] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [term 3 LEADER]: Becoming Leader. State: Replica: 02ee5b2754924933997a4ba119c02639, State: Running, Role: LEADER
I20260925 22:05:41.368945 12540 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: false } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: false } }
I20260925 22:05:41.369298 12543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02ee5b2754924933997a4ba119c02639. Latest consensus state: current_term: 3 leader_uuid: "02ee5b2754924933997a4ba119c02639" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: false } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: false } } }
I20260925 22:05:41.369402 12543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:41.369616 12545 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:05:41.370046 12545 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:05:41.370154 12545 catalog_manager.cc:1259] Loaded cluster ID: 812165beba8f4c9c8ac57ab50e1ed30e
I20260925 22:05:41.370189 12545 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:05:41.370416 12545 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:05:41.370520 12545 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639: Loaded TSK: 0
I20260925 22:05:41.370661 12545 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:05:41.457835 11994 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45560:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:05:41.459489 11994 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:05:41.460536 12541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:41.460426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.0.0.1:42590)
0925 22:05:41.460474 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:41.460508 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260925 22:05:41.460613 11825 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [term 3 FOLLOWER]: Refusing update from remote peer 02ee5b2754924933997a4ba119c02639: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260925 22:05:41.460923 12541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:41.460826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.0.0.1:42598)
0925 22:05:41.460876 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:41.460906 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260925 22:05:41.460878 12543 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260925 22:05:41.461014 11988 consensus_peers.cc:487] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 -> Peer 8faad6f3402b4d1794676034a93d324a (127.5.0.126:41553): Couldn't send request to peer 8faad6f3402b4d1794676034a93d324a. Status: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 22:05:41.461592 12547 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02ee5b2754924933997a4ba119c02639. Latest consensus state: current_term: 3 leader_uuid: "02ee5b2754924933997a4ba119c02639" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: false } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: false } } }
I20260925 22:05:41.461692 12547 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:05:41.462051 12546 mvcc.cc:205] Tried to move back new op lower bound from 7333371664220397568 to 7333371663847862272. Current Snapshot: MvccSnapshot[applied={T|T < 7333371664220397568}]
I20260925 22:05:41.462141 12540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "02ee5b2754924933997a4ba119c02639" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: false } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: false } } }
I20260925 22:05:41.462246 12540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639 [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:41.462244 12548 mvcc.cc:205] Tried to move back new op lower bound from 7333371664220397568 to 7333371663847862272. Current Snapshot: MvccSnapshot[applied={T|T < 7333371664220397568}]
I20260925 22:05:41.462332 12547 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "02ee5b2754924933997a4ba119c02639" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8faad6f3402b4d1794676034a93d324a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41553 } } peers { permanent_uuid: "f6d92dd7ba964ba6b2f2b1a84b443f9f" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 40765 } attrs { promote: false } } peers { permanent_uuid: "02ee5b2754924933997a4ba119c02639" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 40507 } attrs { promote: false } } }
I20260925 22:05:41.462419 12547 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6d92dd7ba964ba6b2f2b1a84b443f9f [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:05:41.467574 12556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:41.464327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47138 (local address 127.5.0.66:34921)
0925 22:05:41.464492 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:41.464495 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:41.464502 (+ 7us) server_negotiation.cc:407] Connection header received
0925 22:05:41.464528 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:41.464529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:41.464540 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:41.464605 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:41.464848 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:41.465377 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:41.466657 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:41.467206 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:41.467284 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:41.467295 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:41.467457 (+ 162us) server_negotiation.cc:299] Negotiation successful
0925 22:05:41.467494 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":52,"threads_started":1}
I20260925 22:05:41.467608 12541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:41.464129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:34921 (local address 127.0.0.1:47138)
0925 22:05:41.464176 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:41.464187 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:41.464237 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:41.464604 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:41.464605 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:41.464606 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:41.464661 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:41.464664 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:41.465462 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:41.465463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:41.466558 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:41.466561 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:41.467299 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:41.467301 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:41.467375 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:41.467379 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:05:41.467445 (+ 66us) client_negotiation.cc:240] Negotiation successful
0925 22:05:41.467523 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260925 22:05:41.467947 12555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:41.464251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:44879 (local address 127.0.0.1:35744)
0925 22:05:41.464654 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:41.464663 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:41.464717 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:41.464845 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:41.464846 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:41.464847 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:41.464881 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:41.464882 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:41.465820 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:41.465821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:41.466866 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:41.466870 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:41.467679 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:41.467681 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:41.467760 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:41.467763 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:05:41.467810 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 22:05:41.467839 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":1,"thread_start_us":65,"threads_started":1}
I20260925 22:05:41.467978 12557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:41.464470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35744 (local address 127.5.0.65:44879)
0925 22:05:41.464728 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:41.464730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:41.464740 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:05:41.464771 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:41.464772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:41.464781 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:41.464846 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:41.464931 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:41.465727 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:41.466999 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:41.467560 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:41.467631 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:41.467643 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:41.467837 (+ 194us) server_negotiation.cc:299] Negotiation successful
0925 22:05:41.467905 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":55,"threads_started":1}
I20260925 22:05:41.468191 12115 tablet_service.cc:1467] Processing CreateTablet for tablet df81e0d00346409085f69edc9bb368ad (DEFAULT_TABLE table=test-table [id=7f195952193e4d13aa54fb4e20b658b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:41.468204 12252 tablet_service.cc:1467] Processing CreateTablet for tablet df81e0d00346409085f69edc9bb368ad (DEFAULT_TABLE table=test-table [id=7f195952193e4d13aa54fb4e20b658b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:41.468324 12115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df81e0d00346409085f69edc9bb368ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:41.468364 12252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df81e0d00346409085f69edc9bb368ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:41.468551 12554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:41.464256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45838 (local address 127.5.0.67:32981)
0925 22:05:41.464421 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:41.464423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:41.464479 (+ 56us) server_negotiation.cc:407] Connection header received
0925 22:05:41.464561 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:41.464563 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:41.464574 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:41.464648 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:41.464830 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:41.465683 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:41.466979 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:41.467576 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:41.467949 (+ 373us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:41.467970 (+ 21us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:41.468378 (+ 408us) server_negotiation.cc:299] Negotiation successful
0925 22:05:41.468438 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":56,"threads_started":1}
I20260925 22:05:41.468740 12553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:41.464197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:32981 (local address 127.0.0.1:45838)
0925 22:05:41.464422 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:41.464431 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:41.464492 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:41.464736 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:41.464736 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:41.464738 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:41.464772 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:41.464773 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:41.465772 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:41.465773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:41.466888 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:41.466890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:41.468192 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:41.468193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:41.468268 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:41.468273 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:05:41.468632 (+ 359us) client_negotiation.cc:240] Negotiation successful
0925 22:05:41.468677 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":24,"thread_start_us":78,"threads_started":1}
I20260925 22:05:41.469545 12558 tablet_bootstrap.cc:492] T df81e0d00346409085f69edc9bb368ad P 2d23d1eb9a374b9a87383727acd62f84: Bootstrap starting.
I20260925 22:05:41.469632 12559 tablet_bootstrap.cc:492] T df81e0d00346409085f69edc9bb368ad P e74b9533a2084cb5a37834fc583b7beb: Bootstrap starting.
I20260925 22:05:41.469964 12388 tablet_service.cc:1467] Processing CreateTablet for tablet df81e0d00346409085f69edc9bb368ad (DEFAULT_TABLE table=test-table [id=7f195952193e4d13aa54fb4e20b658b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:41.470037 12559 tablet_bootstrap.cc:654] T df81e0d00346409085f69edc9bb368ad P e74b9533a2084cb5a37834fc583b7beb: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:41.470099 12388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df81e0d00346409085f69edc9bb368ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:41.470121 12558 tablet_bootstrap.cc:654] T df81e0d00346409085f69edc9bb368ad P 2d23d1eb9a374b9a87383727acd62f84: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:41.470709 12559 tablet_bootstrap.cc:492] T df81e0d00346409085f69edc9bb368ad P e74b9533a2084cb5a37834fc583b7beb: No bootstrap required, opened a new log
I20260925 22:05:41.470767 12558 tablet_bootstrap.cc:492] T df81e0d00346409085f69edc9bb368ad P 2d23d1eb9a374b9a87383727acd62f84: No bootstrap required, opened a new log
I20260925 22:05:41.470774 12559 ts_tablet_manager.cc:1397] T df81e0d00346409085f69edc9bb368ad P e74b9533a2084cb5a37834fc583b7beb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 22:05:41.470834 12558 ts_tablet_manager.cc:1397] T df81e0d00346409085f69edc9bb368ad P 2d23d1eb9a374b9a87383727acd62f84: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 22:05:41.470916 12559 raft_consensus.cc:348] T df81e0d00346409085f69edc9bb368ad P e74b9533a2084cb5a37834fc583b7beb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e74b9533a2084cb5a37834fc583b7beb" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 34921 } } peers { permanent_uuid: "2d23d1eb9a374b9a87383727acd62f84" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 44879 } } peers { permanent_uuid: "35c4eb82e71d482aac5e40b60a85dcbc" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32981 } }
I20260925 22:05:41.471009 12559 raft_consensus.cc:374] T df81e0d00346409085f69edc9bb368ad P e74b9533a2084cb5a37834fc583b7beb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:41.471040 12559 raft_consensus.cc:729] T df81e0d00346409085f69edc9bb368ad P e74b9533a2084cb5a37834fc583b7beb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e74b9533a2084cb5a37834fc583b7beb, State: Initialized, Role: FOLLOWER
I20260925 22:05:41.470983 12558 raft_consensus.cc:348] T df81e0d00346409085f69edc9bb368ad P 2d23d1eb9a374b9a87383727acd62f84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e74b9533a2084cb5a37834fc583b7beb" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 34921 } } peers { permanent_uuid: "2d23d1eb9a374b9a87383727acd62f84" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 44879 } } peers { permanent_uuid: "35c4eb82e71d482aac5e40b60a85dcbc" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32981 } }
I20260925 22:05:41.471072 12558 raft_consensus.cc:374] T df81e0d00346409085f69edc9bb368ad P 2d23d1eb9a374b9a87383727acd62f84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:41.471079 12559 consensus_queue.cc:260] T df81e0d00346409085f69edc9bb368ad P e74b9533a2084cb5a37834fc583b7beb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e74b9533a2084cb5a37834fc583b7beb" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 34921 } } peers { permanent_uuid: "2d23d1eb9a374b9a87383727acd62f84" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 44879 } } peers { permanent_uuid: "35c4eb82e71d482aac5e40b60a85dcbc" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32981 } }
I20260925 22:05:41.471108 12558 raft_consensus.cc:729] T df81e0d00346409085f69edc9bb368ad P 2d23d1eb9a374b9a87383727acd62f84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d23d1eb9a374b9a87383727acd62f84, State: Initialized, Role: FOLLOWER
I20260925 22:05:41.471153 12558 consensus_queue.cc:260] T df81e0d00346409085f69edc9bb368ad P 2d23d1eb9a374b9a87383727acd62f84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e74b9533a2084cb5a37834fc583b7beb" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 34921 } } peers { permanent_uuid: "2d23d1eb9a374b9a87383727acd62f84" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 44879 } } peers { permanent_uuid: "35c4eb82e71d482aac5e40b60a85dcbc" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32981 } }
I20260925 22:05:41.471264 12559 ts_tablet_manager.cc:1428] T df81e0d00346409085f69edc9bb368ad P e74b9533a2084cb5a37834fc583b7beb: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 22:05:41.471323 12558 ts_tablet_manager.cc:1428] T df81e0d00346409085f69edc9bb368ad P 2d23d1eb9a374b9a87383727acd62f84: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 22:05:41.471720 12564 tablet_bootstrap.cc:492] T df81e0d00346409085f69edc9bb368ad P 35c4eb82e71d482aac5e40b60a85dcbc: Bootstrap starting.
I20260925 22:05:41.472048 12538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:41.471651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.5.0.65:47761)
0925 22:05:41.471976 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:41.472012 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296}
I20260925 22:05:41.472050 12537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:41.471960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.5.0.66:38313)
0925 22:05:41.471997 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:41.472023 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260925 22:05:41.472146 12564 tablet_bootstrap.cc:654] T df81e0d00346409085f69edc9bb368ad P 35c4eb82e71d482aac5e40b60a85dcbc: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:41.472625 12318 heartbeater.cc:503] Master 127.5.0.124:40507 was elected leader, sending a full tablet report...
I20260925 22:05:41.473009 12537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:41.472893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.5.0.66:45363)
0925 22:05:41.472930 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:41.472979 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260925 22:05:41.473048 12538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:41.472960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.5.0.65:43857)
0925 22:05:41.472997 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:41.473018 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260925 22:05:41.473097 12181 heartbeater.cc:503] Master 127.5.0.124:40507 was elected leader, sending a full tablet report...
I20260925 22:05:41.473533 12564 tablet_bootstrap.cc:492] T df81e0d00346409085f69edc9bb368ad P 35c4eb82e71d482aac5e40b60a85dcbc: No bootstrap required, opened a new log
I20260925 22:05:41.473593 12564 ts_tablet_manager.cc:1397] T df81e0d00346409085f69edc9bb368ad P 35c4eb82e71d482aac5e40b60a85dcbc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:05:41.473728 12564 raft_consensus.cc:348] T df81e0d00346409085f69edc9bb368ad P 35c4eb82e71d482aac5e40b60a85dcbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e74b9533a2084cb5a37834fc583b7beb" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 34921 } } peers { permanent_uuid: "2d23d1eb9a374b9a87383727acd62f84" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 44879 } } peers { permanent_uuid: "35c4eb82e71d482aac5e40b60a85dcbc" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32981 } }
I20260925 22:05:41.473780 12564 raft_consensus.cc:374] T df81e0d00346409085f69edc9bb368ad P 35c4eb82e71d482aac5e40b60a85dcbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:41.473794 12564 raft_consensus.cc:729] T df81e0d00346409085f69edc9bb368ad P 35c4eb82e71d482aac5e40b60a85dcbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35c4eb82e71d482aac5e40b60a85dcbc, State: Initialized, Role: FOLLOWER
I20260925 22:05:41.473861 12564 consensus_queue.cc:260] T df81e0d00346409085f69edc9bb368ad P 35c4eb82e71d482aac5e40b60a85dcbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e74b9533a2084cb5a37834fc583b7beb" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 34921 } } peers { permanent_uuid: "2d23d1eb9a374b9a87383727acd62f84" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 44879 } } peers { permanent_uuid: "35c4eb82e71d482aac5e40b60a85dcbc" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32981 } }
I20260925 22:05:41.474015 12564 ts_tablet_manager.cc:1428] T df81e0d00346409085f69edc9bb368ad P 35c4eb82e71d482aac5e40b60a85dcbc: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 22:05:41.474274 12539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:41.474174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.5.0.67:33197)
0925 22:05:41.474210 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:41.474244 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260925 22:05:41.474573 12454 heartbeater.cc:503] Master 127.5.0.124:40507 was elected leader, sending a full tablet report...
I20260925 22:05:41.474680 12539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:41.474570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.5.0.67:45145)
0925 22:05:41.474621 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:41.474652 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260925 22:05:41.675458 12563 raft_consensus.cc:482] T df81e0d00346409085f69edc9bb368ad P 2d23d1eb9a374b9a87383727acd62f84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:05:41.675589 12563 raft_consensus.cc:504] T df81e0d00346409085f69edc9bb368ad P 2d23d1eb9a374b9a87383727acd62f84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e74b9533a2084cb5a37834fc583b7beb" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 34921 } } peers { permanent_uuid: "2d23d1eb9a374b9a87383727acd62f84" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 44879 } } peers { permanent_uuid: "35c4eb82e71d482aac5e40b60a85dcbc" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32981 } }
I20260925 22:05:41.675830 12563 leader_election.cc:288] T df81e0d00346409085f69edc9bb368ad P 2d23d1eb9a374b9a87383727acd62f84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e74b9533a2084cb5a37834fc583b7beb (127.5.0.66:34921), 35c4eb82e71d482aac5e40b60a85dcbc (127.5.0.67:32981)
I20260925 22:05:41.676079 12272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df81e0d00346409085f69edc9bb368ad" candidate_uuid: "2d23d1eb9a374b9a87383727acd62f84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e74b9533a2084cb5a37834fc583b7beb" is_pre_election: true
I20260925 22:05:41.676157 12272 raft_consensus.cc:2454] T df81e0d00346409085f69edc9bb368ad P e74b9533a2084cb5a37834fc583b7beb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d23d1eb9a374b9a87383727acd62f84 in term 0.
I20260925 22:05:41.676074 12408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df81e0d00346409085f69edc9bb368ad" candidate_uuid: "2d23d1eb9a374b9a87383727acd62f84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35c4eb82e71d482aac5e40b60a85dcbc" is_pre_election: true
I20260925 22:05:41.676195 12408 raft_consensus.cc:2454] T df81e0d00346409085f69edc9bb368ad P 35c4eb82e71d482aac5e40b60a85dcbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d23d1eb9a374b9a87383727acd62f84 in term 0.
I20260925 22:05:41.676327 12068 leader_election.cc:302] T df81e0d00346409085f69edc9bb368ad P 2d23d1eb9a374b9a87383727acd62f84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d23d1eb9a374b9a87383727acd62f84, e74b9533a2084cb5a37834fc583b7beb; no voters:
I20260925 22:05:41.676440 12563 raft_consensus.cc:2781] T df81e0d00346409085f69edc9bb368ad P 2d23d1eb9a374b9a87383727acd62f84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:05:41.676487 12563 raft_consensus.cc:482] T df81e0d00346409085f69edc9bb368ad P 2d23d1eb9a374b9a87383727acd62f84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:05:41.676517 12563 raft_consensus.cc:3037] T df81e0d00346409085f69edc9bb368ad P 2d23d1eb9a374b9a87383727acd62f84 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:41.677268 12563 raft_consensus.cc:504] T df81e0d00346409085f69edc9bb368ad P 2d23d1eb9a374b9a87383727acd62f84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e74b9533a2084cb5a37834fc583b7beb" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 34921 } } peers { permanent_uuid: "2d23d1eb9a374b9a87383727acd62f84" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 44879 } } peers { permanent_uuid: "35c4eb82e71d482aac5e40b60a85dcbc" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32981 } }
I20260925 22:05:41.677410 12563 leader_election.cc:288] T df81e0d00346409085f69edc9bb368ad P 2d23d1eb9a374b9a87383727acd62f84 [CANDIDATE]: Term 1 election: Requested vote from peers e74b9533a2084cb5a37834fc583b7beb (127.5.0.66:34921), 35c4eb82e71d482aac5e40b60a85dcbc (127.5.0.67:32981)
I20260925 22:05:41.677568 12408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df81e0d00346409085f69edc9bb368ad" candidate_uuid: "2d23d1eb9a374b9a87383727acd62f84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35c4eb82e71d482aac5e40b60a85dcbc"
I20260925 22:05:41.677562 12272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df81e0d00346409085f69edc9bb368ad" candidate_uuid: "2d23d1eb9a374b9a87383727acd62f84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e74b9533a2084cb5a37834fc583b7beb"
I20260925 22:05:41.677644 12272 raft_consensus.cc:3037] T df81e0d00346409085f69edc9bb368ad P e74b9533a2084cb5a37834fc583b7beb [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:41.677665 12408 raft_consensus.cc:3037] T df81e0d00346409085f69edc9bb368ad P 35c4eb82e71d482aac5e40b60a85dcbc [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:41.678295 12272 raft_consensus.cc:2454] T df81e0d00346409085f69edc9bb368ad P e74b9533a2084cb5a37834fc583b7beb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d23d1eb9a374b9a87383727acd62f84 in term 1.
I20260925 22:05:41.678340 12408 raft_consensus.cc:2454] T df81e0d00346409085f69edc9bb368ad P 35c4eb82e71d482aac5e40b60a85dcbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d23d1eb9a374b9a87383727acd62f84 in term 1.
I20260925 22:05:41.678457 12068 leader_election.cc:302] T df81e0d00346409085f69edc9bb368ad P 2d23d1eb9a374b9a87383727acd62f84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d23d1eb9a374b9a87383727acd62f84, e74b9533a2084cb5a37834fc583b7beb; no voters:
I20260925 22:05:41.678578 12563 raft_consensus.cc:2781] T df81e0d00346409085f69edc9bb368ad P 2d23d1eb9a374b9a87383727acd62f84 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:41.678714 12563 raft_consensus.cc:686] T df81e0d00346409085f69edc9bb368ad P 2d23d1eb9a374b9a87383727acd62f84 [term 1 LEADER]: Becoming Leader. State: Replica: 2d23d1eb9a374b9a87383727acd62f84, State: Running, Role: LEADER
I20260925 22:05:41.678756 12563 consensus_queue.cc:237] T df81e0d00346409085f69edc9bb368ad P 2d23d1eb9a374b9a87383727acd62f84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e74b9533a2084cb5a37834fc583b7beb" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 34921 } } peers { permanent_uuid: "2d23d1eb9a374b9a87383727acd62f84" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 44879 } } peers { permanent_uuid: "35c4eb82e71d482aac5e40b60a85dcbc" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32981 } }
I20260925 22:05:41.679064 12538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:41.678960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.5.0.65:43811)
0925 22:05:41.678999 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:41.679039 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260925 22:05:41.679426 11994 catalog_manager.cc:5495] T df81e0d00346409085f69edc9bb368ad P 2d23d1eb9a374b9a87383727acd62f84 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d23d1eb9a374b9a87383727acd62f84 (127.5.0.65). New cstate: current_term: 1 leader_uuid: "2d23d1eb9a374b9a87383727acd62f84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e74b9533a2084cb5a37834fc583b7beb" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 34921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d23d1eb9a374b9a87383727acd62f84" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 44879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35c4eb82e71d482aac5e40b60a85dcbc" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32981 } health_report { overall_health: UNKNOWN } } }
I20260925 22:05:41.694024 12557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:41.692083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35758 (local address 127.5.0.65:44879)
0925 22:05:41.692142 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:41.692148 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:41.692157 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:05:41.692179 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:41.692180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:41.692194 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:41.692239 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:05:41.692360 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:41.692826 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:41.693502 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:41.693687 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:41.693733 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:41.693843 (+ 110us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:05:41.693867 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:05:41.693903 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:41.693954 (+ 51us) server_negotiation.cc:299] Negotiation successful
0925 22:05:41.693970 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 22:05:41.694918 12408 raft_consensus.cc:1261] T df81e0d00346409085f69edc9bb368ad P 35c4eb82e71d482aac5e40b60a85dcbc [term 1 FOLLOWER]: Refusing update from remote peer 2d23d1eb9a374b9a87383727acd62f84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:05:41.694957 12272 raft_consensus.cc:1261] T df81e0d00346409085f69edc9bb368ad P e74b9533a2084cb5a37834fc583b7beb [term 1 FOLLOWER]: Refusing update from remote peer 2d23d1eb9a374b9a87383727acd62f84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:05:41.695137 12567 consensus_queue.cc:1035] T df81e0d00346409085f69edc9bb368ad P 2d23d1eb9a374b9a87383727acd62f84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35c4eb82e71d482aac5e40b60a85dcbc" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:05:41.695286 12563 consensus_queue.cc:1035] T df81e0d00346409085f69edc9bb368ad P 2d23d1eb9a374b9a87383727acd62f84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e74b9533a2084cb5a37834fc583b7beb" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 34921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:05:41.695757 12539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:41.695644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.5.0.67:39755)
0925 22:05:41.695682 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:41.695726 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260925 22:05:41.696372 12537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:41.696217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.5.0.66:54983)
0925 22:05:41.696304 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:41.696339 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260925 22:05:41.696686 12570 mvcc.cc:205] Tried to move back new op lower bound from 7333371665180549120 to 7333371665116770304. Current Snapshot: MvccSnapshot[applied={T|T < 7333371665180549120}]
I20260925 22:05:41.697072 12538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:41.696981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.5.0.65:42735)
0925 22:05:41.697016 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:41.697044 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260925 22:05:41.697754 12539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:41.697650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.5.0.67:38751)
0925 22:05:41.697688 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:41.697725 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260925 22:05:41.698062 12538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:41.697859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.5.0.65:55635)
0925 22:05:41.698008 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:41.698035 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
I20260925 22:05:41.698191 12537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:41.698090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41553 (local address 127.5.0.66:45395)
0925 22:05:41.698131 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:41.698154 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260925 22:05:41.698944 11993 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:05:41.699023 11993 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:05:41.700223 11993 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 02ee5b2754924933997a4ba119c02639: Sending DeleteTablet for 3 replicas of tablet df81e0d00346409085f69edc9bb368ad
I20260925 22:05:41.700472 12252 tablet_service.cc:1514] Processing DeleteTablet for tablet df81e0d00346409085f69edc9bb368ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:41 UTC) from {username='oryx', principal='oryx/127.5.0.124@KRBTEST.COM'} at 127.0.0.1:47138
I20260925 22:05:41.700500 12115 tablet_service.cc:1514] Processing DeleteTablet for tablet df81e0d00346409085f69edc9bb368ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:41 UTC) from {username='oryx', principal='oryx/127.5.0.124@KRBTEST.COM'} at 127.0.0.1:35744
I20260925 22:05:41.700558 12388 tablet_service.cc:1514] Processing DeleteTablet for tablet df81e0d00346409085f69edc9bb368ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:41 UTC) from {username='oryx', principal='oryx/127.5.0.124@KRBTEST.COM'} at 127.0.0.1:45838
I20260925 22:05:41.700688 12578 tablet_replica.cc:330] T df81e0d00346409085f69edc9bb368ad P e74b9533a2084cb5a37834fc583b7beb: stopping tablet replica
I20260925 22:05:41.700757 12578 raft_consensus.cc:2229] T df81e0d00346409085f69edc9bb368ad P e74b9533a2084cb5a37834fc583b7beb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:05:41.700810 12578 raft_consensus.cc:2258] T df81e0d00346409085f69edc9bb368ad P e74b9533a2084cb5a37834fc583b7beb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:41.700814 12580 tablet_replica.cc:330] T df81e0d00346409085f69edc9bb368ad P 35c4eb82e71d482aac5e40b60a85dcbc: stopping tablet replica
I20260925 22:05:41.700860 12580 raft_consensus.cc:2229] T df81e0d00346409085f69edc9bb368ad P 35c4eb82e71d482aac5e40b60a85dcbc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:05:41.700908 12580 raft_consensus.cc:2258] T df81e0d00346409085f69edc9bb368ad P 35c4eb82e71d482aac5e40b60a85dcbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:41.701009 12579 tablet_replica.cc:330] T df81e0d00346409085f69edc9bb368ad P 2d23d1eb9a374b9a87383727acd62f84: stopping tablet replica
I20260925 22:05:41.701014 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 11963
I20260925 22:05:41.701059 12579 raft_consensus.cc:2229] T df81e0d00346409085f69edc9bb368ad P 2d23d1eb9a374b9a87383727acd62f84 [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:05:41.701121 12579 raft_consensus.cc:2258] T df81e0d00346409085f69edc9bb368ad P 2d23d1eb9a374b9a87383727acd62f84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:41.701169 12578 ts_tablet_manager.cc:1905] T df81e0d00346409085f69edc9bb368ad P e74b9533a2084cb5a37834fc583b7beb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:41.701740 12580 ts_tablet_manager.cc:1905] T df81e0d00346409085f69edc9bb368ad P 35c4eb82e71d482aac5e40b60a85dcbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:41.703178 12580 ts_tablet_manager.cc:1918] T df81e0d00346409085f69edc9bb368ad P 35c4eb82e71d482aac5e40b60a85dcbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:05:41.703244 12580 log.cc:1196] T df81e0d00346409085f69edc9bb368ad P 35c4eb82e71d482aac5e40b60a85dcbc: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-2/wal/wals/df81e0d00346409085f69edc9bb368ad
I20260925 22:05:41.703498 12580 ts_tablet_manager.cc:1939] T df81e0d00346409085f69edc9bb368ad P 35c4eb82e71d482aac5e40b60a85dcbc: Deleting consensus metadata
I20260925 22:05:41.703776 12578 ts_tablet_manager.cc:1918] T df81e0d00346409085f69edc9bb368ad P e74b9533a2084cb5a37834fc583b7beb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:05:41.703840 12578 log.cc:1196] T df81e0d00346409085f69edc9bb368ad P e74b9533a2084cb5a37834fc583b7beb: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373908987421-5121-0/minicluster-data/ts-1/wal/wals/df81e0d00346409085f69edc9bb368ad
I20260925 22:05:41.703835 11989 catalog_manager.cc:4841] TS 35c4eb82e71d482aac5e40b60a85dcbc (127.5.0.67:32981): tablet df81e0d00346409085f69edc9bb368ad (table test-table [id=7f195952193e4d13aa54fb4e20b658b9]) successfully deleted
I20260925 22:05:41.704072 12578 ts_tablet_manager.cc:1939] T df81e0d00346409085f69edc9bb368ad P e74b9533a2084cb5a37834fc583b7beb: Deleting consensus metadata
I20260925 22:05:41.704378 11986 catalog_manager.cc:4841] TS e74b9533a2084cb5a37834fc583b7beb (127.5.0.66:34921): tablet df81e0d00346409085f69edc9bb368ad (table test-table [id=7f195952193e4d13aa54fb4e20b658b9]) successfully deleted
W20260925 22:05:41.708557 12553 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:41.708436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:44879 (local address 127.0.0.1:35772)
0925 22:05:41.708471 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:41.708498 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.65:44879: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":19}
W20260925 22:05:41.708724 11986 catalog_manager.cc:4584] TS 2d23d1eb9a374b9a87383727acd62f84 (127.5.0.65:44879): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet df81e0d00346409085f69edc9bb368ad: Network error: Client connection negotiation failed: client connection to 127.5.0.65:44879: connect: Connection reset by peer (error 104)
I20260925 22:05:41.708746 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 12189
I20260925 22:05:41.714013 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 12325
I20260925 22:05:41.719528 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 11681
I20260925 22:05:41.724299 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 11870
2026-09-25T22:05:41Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (8249 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:05:41 dist-test-slave-7mqr krb5kdc[12583](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:05:41 dist-test-slave-7mqr krb5kdc[12583](info): set up 2 sockets
Sep 25 22:05:41 dist-test-slave-7mqr krb5kdc[12583](info): commencing operation
krb5kdc: starting...
W20260925 22:05:42.760352 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:05:42 dist-test-slave-7mqr krb5kdc[12583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:05:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:05:42Z Disabled control of system clock
WARNING: no policy specified for oryx/127.5.0.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.0.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.0.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.126.keytab.
Entry for principal oryx/127.5.0.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.126.keytab.
I20260925 22:05:42.876682 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:45363
--webserver_interface=127.5.0.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.126.keytab
--principal=oryx/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:42003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:45363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:42.957276 12596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:42.957470 12596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:42.957492 12596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:42.958554 12596 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:05:42.958585 12596 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 22:05:42.958597 12596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:42.958611 12596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:42.958622 12596 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:05:42.958634 12596 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:05:42.960095 12596 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:42003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:45363
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.126.keytab
--principal=oryx/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:45363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:42.960280 12596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:42.960444 12596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:42.964986 12602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:42.964994 12601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:42.965296 12596 server_base.cc:1034] running on GCE node
W20260925 22:05:42.964994 12604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:42.965678 12596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:42.965956 12596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:42.967119 12596 hybrid_clock.cc:648] HybridClock initialized: now 1790373942967112 us; error 35 us; skew 500 ppm
Sep 25 22:05:42 dist-test-slave-7mqr krb5kdc[12583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373942, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:42.969504 12596 init.cc:377] Logged in from keytab as oryx/127.5.0.126@KRBTEST.COM (short username oryx)
I20260925 22:05:42.969954 12596 webserver.cc:460] Webserver started at http://127.5.0.126:33891/ using document root <none> and password file <none>
I20260925 22:05:42.970194 12596 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:42.970263 12596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:42.970412 12596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:42.971431 12596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "f5a748e94d32429ca130f24868a54d8c"
format_stamp: "Formatted at 2026-09-25 22:05:42 on dist-test-slave-7mqr"
server_key: "794ffd3c831bbbdfd868f630d1b40990"
server_key_iv: "4d9177fc7775e11f08668a5e1861722a"
server_key_version: "encryptionkey@0"
I20260925 22:05:42.971786 12596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "f5a748e94d32429ca130f24868a54d8c"
format_stamp: "Formatted at 2026-09-25 22:05:42 on dist-test-slave-7mqr"
server_key: "794ffd3c831bbbdfd868f630d1b40990"
server_key_iv: "4d9177fc7775e11f08668a5e1861722a"
server_key_version: "encryptionkey@0"
I20260925 22:05:42.973070 12596 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:05:42.973762 12611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:42.973919 12596 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:05:42.974016 12596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "f5a748e94d32429ca130f24868a54d8c"
format_stamp: "Formatted at 2026-09-25 22:05:42 on dist-test-slave-7mqr"
server_key: "794ffd3c831bbbdfd868f630d1b40990"
server_key_iv: "4d9177fc7775e11f08668a5e1861722a"
server_key_version: "encryptionkey@0"
I20260925 22:05:42.974094 12596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:42.983467 12596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:42.985559 12596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:42.985720 12596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:42.990425 12596 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:45363
I20260925 22:05:42.990473 12673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:45363 every 8 connection(s)
I20260925 22:05:42.990831 12596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:05:42.991109 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 12596
I20260925 22:05:42.991216 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:05:42.991385 5121 external_mini_cluster.cc:1428] Setting key 5365d716a93191f5f242dc1afb9e23ba
I20260925 22:05:42.991600 12674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:42.994810 12674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5a748e94d32429ca130f24868a54d8c: Bootstrap starting.
I20260925 22:05:42.995517 12674 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5a748e94d32429ca130f24868a54d8c: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:42.995810 12674 log.cc:824] T 00000000000000000000000000000000 P f5a748e94d32429ca130f24868a54d8c: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:42.996616 12674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5a748e94d32429ca130f24868a54d8c: No bootstrap required, opened a new log
Sep 25 22:05:42 dist-test-slave-7mqr krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM
I20260925 22:05:42.998976 12674 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5a748e94d32429ca130f24868a54d8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5a748e94d32429ca130f24868a54d8c" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 45363 } }
I20260925 22:05:42.999117 12674 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5a748e94d32429ca130f24868a54d8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:42.999140 12674 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5a748e94d32429ca130f24868a54d8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5a748e94d32429ca130f24868a54d8c, State: Initialized, Role: FOLLOWER
I20260925 22:05:42.999320 12674 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5a748e94d32429ca130f24868a54d8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5a748e94d32429ca130f24868a54d8c" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 45363 } }
I20260925 22:05:42.999413 12674 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5a748e94d32429ca130f24868a54d8c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:05:42.999449 12674 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5a748e94d32429ca130f24868a54d8c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:05:42.999504 12674 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5a748e94d32429ca130f24868a54d8c [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:43.000219 12674 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5a748e94d32429ca130f24868a54d8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5a748e94d32429ca130f24868a54d8c" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 45363 } }
I20260925 22:05:43.000339 12674 leader_election.cc:302] T 00000000000000000000000000000000 P f5a748e94d32429ca130f24868a54d8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5a748e94d32429ca130f24868a54d8c; no voters:
I20260925 22:05:43.000487 12674 leader_election.cc:288] T 00000000000000000000000000000000 P f5a748e94d32429ca130f24868a54d8c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:05:43.000557 12679 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5a748e94d32429ca130f24868a54d8c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:43.000708 12679 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5a748e94d32429ca130f24868a54d8c [term 1 LEADER]: Becoming Leader. State: Replica: f5a748e94d32429ca130f24868a54d8c, State: Running, Role: LEADER
I20260925 22:05:43.000741 12674 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5a748e94d32429ca130f24868a54d8c [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:05:43.000792 12679 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5a748e94d32429ca130f24868a54d8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5a748e94d32429ca130f24868a54d8c" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 45363 } }
I20260925 22:05:43.001209 12680 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5a748e94d32429ca130f24868a54d8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5a748e94d32429ca130f24868a54d8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5a748e94d32429ca130f24868a54d8c" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 45363 } } }
I20260925 22:05:43.001322 12680 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5a748e94d32429ca130f24868a54d8c [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:43.001205 12681 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5a748e94d32429ca130f24868a54d8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5a748e94d32429ca130f24868a54d8c. Latest consensus state: current_term: 1 leader_uuid: "f5a748e94d32429ca130f24868a54d8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5a748e94d32429ca130f24868a54d8c" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 45363 } } }
I20260925 22:05:43.001478 12681 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5a748e94d32429ca130f24868a54d8c [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:43.001583 12687 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:05:43.002084 12687 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:05:43.002749 12677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:42.991879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35094 (local address 127.5.0.126:45363)
0925 22:05:42.992069 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:42.992078 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:42.992112 (+ 34us) server_negotiation.cc:407] Connection header received
0925 22:05:42.992734 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:42.992740 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:42.992786 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:42.992897 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:42.993696 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:42.994566 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:42.995609 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:42.996147 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:42.996267 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:42.998808 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:42.998812 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:42.998812 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:42.998822 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:43.000881 (+ 2059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:43.001320 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:43.001323 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:43.001324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:43.001446 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:43.001667 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:43.001667 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:43.001668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:43.002024 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:43.002175 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:43.002410 (+ 235us) server_negotiation.cc:299] Negotiation successful
0925 22:05:43.002544 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":94,"threads_started":1}
I20260925 22:05:43.003579 12687 catalog_manager.cc:1347] Generated new cluster ID: 091da1c59e1a4da19bb0b7cba4f328a0
I20260925 22:05:43.003628 12687 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:05:43.011353 12687 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:05:43.011874 12687 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:05:43.018888 12687 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5a748e94d32429ca130f24868a54d8c: Generated new TSK 0
I20260925 22:05:43.019062 12687 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.5.0.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.0.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.0.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.65.keytab.
Entry for principal oryx/127.5.0.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.65.keytab.
I20260925 22:05:43.052505 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.65:0
--local_ip_for_outbound_sockets=127.5.0.65
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:45363
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.65.keytab
--principal=oryx/127.5.0.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:42003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:43.131846 12700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:43.132113 12700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:43.132138 12700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:43.133255 12700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:43.133287 12700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:43.133307 12700 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 22:05:43.133325 12700 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.65
I20260925 22:05:43.134729 12700 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:42003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.65.keytab
--principal=oryx/127.5.0.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.0.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.0.126:45363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.65
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:43.134902 12700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:43.135095 12700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:43.139858 12708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:43.139873 12700 server_base.cc:1034] running on GCE node
W20260925 22:05:43.139917 12705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:43.139858 12706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:43.140306 12700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:43.140508 12700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:43.141650 12700 hybrid_clock.cc:648] HybridClock initialized: now 1790373943141638 us; error 26 us; skew 500 ppm
Sep 25 22:05:43 dist-test-slave-7mqr krb5kdc[12583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373943, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:43.144127 12700 init.cc:377] Logged in from keytab as oryx/127.5.0.65@KRBTEST.COM (short username oryx)
I20260925 22:05:43.144500 12700 webserver.cc:460] Webserver started at http://127.5.0.65:37329/ using document root <none> and password file <none>
I20260925 22:05:43.144672 12700 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:43.144714 12700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:43.144804 12700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:43.145852 12700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-0/data/instance:
uuid: "87968f19e8d8461aa549ca61f020e42f"
format_stamp: "Formatted at 2026-09-25 22:05:43 on dist-test-slave-7mqr"
server_key: "be7a62622e2a059b618ea96392603bcd"
server_key_iv: "711c3ad5361bca7f6c53507440fee91d"
server_key_version: "encryptionkey@0"
I20260925 22:05:43.146221 12700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance:
uuid: "87968f19e8d8461aa549ca61f020e42f"
format_stamp: "Formatted at 2026-09-25 22:05:43 on dist-test-slave-7mqr"
server_key: "be7a62622e2a059b618ea96392603bcd"
server_key_iv: "711c3ad5361bca7f6c53507440fee91d"
server_key_version: "encryptionkey@0"
I20260925 22:05:43.147533 12700 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:05:43.148257 12715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:43.148422 12700 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:43.148530 12700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-0/wal
uuid: "87968f19e8d8461aa549ca61f020e42f"
format_stamp: "Formatted at 2026-09-25 22:05:43 on dist-test-slave-7mqr"
server_key: "be7a62622e2a059b618ea96392603bcd"
server_key_iv: "711c3ad5361bca7f6c53507440fee91d"
server_key_version: "encryptionkey@0"
I20260925 22:05:43.148602 12700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:43.167995 12700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:43.169838 12700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:43.170022 12700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:43.170568 12700 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:43.170634 12700 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:43.170681 12700 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:43.170708 12700 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:43.177521 12700 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.65:33097
I20260925 22:05:43.177930 12700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
Sep 25 22:05:43 dist-test-slave-7mqr krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373943, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM
I20260925 22:05:43.178732 12830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.65:33097 every 8 connection(s)
I20260925 22:05:43.181197 12677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:43.171123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:60071 (local address 127.5.0.126:45363)
0925 22:05:43.171165 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:43.171167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:43.171419 (+ 252us) server_negotiation.cc:407] Connection header received
0925 22:05:43.172002 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:43.172003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:43.172015 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:43.172077 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:43.172976 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.173847 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.175455 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.175798 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.175905 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:43.178669 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:43.178670 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:43.178670 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:43.178677 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:43.180063 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:43.180422 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:43.180423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:43.180423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:43.180476 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:43.180690 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:43.180690 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:43.180690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:43.180805 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:43.180856 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:43.181062 (+ 206us) server_negotiation.cc:299] Negotiation successful
0925 22:05:43.181131 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 22:05:43.181425 12724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:43.171132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:45363 (local address 127.5.0.65:60071)
0925 22:05:43.171299 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:43.171335 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:43.171852 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:43.172154 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:43.172158 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:43.172554 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:43.172856 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:43.172863 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.173966 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.173967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:43.175353 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:43.175356 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.175947 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.175948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:43.176061 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:43.176713 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:43.176724 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:43.178585 (+ 1861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:43.180134 (+ 1549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:43.180139 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:43.180141 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:43.180359 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:43.180532 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:43.180532 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:43.180533 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:43.180626 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:43.180858 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:43.180861 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:43.180972 (+ 111us) client_negotiation.cc:769] Sending connection context
0925 22:05:43.181052 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 22:05:43.181174 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":86,"threads_started":1}
I20260925 22:05:43.181922 12831 heartbeater.cc:347] Connected to a master server at 127.5.0.126:45363
I20260925 22:05:43.182001 12831 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:43.182197 12831 heartbeater.cc:511] Master 127.5.0.126:45363 requested a full tablet report, sending...
I20260925 22:05:43.182775 12627 ts_manager.cc:194] Registered new tserver with Master: 87968f19e8d8461aa549ca61f020e42f (127.5.0.65:33097)
I20260925 22:05:43.183662 12627 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.0.65@KRBTEST.COM'} at 127.5.0.65:60071
I20260925 22:05:43.187184 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 12700
I20260925 22:05:43.187305 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance
I20260925 22:05:43.187536 5121 external_mini_cluster.cc:1428] Setting key 9450484804002fb14ba48349b84a11e7
I20260925 22:05:43.188627 12677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:43.183080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:49845 (local address 127.5.0.126:45363)
0925 22:05:43.183120 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:43.183121 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:43.183188 (+ 67us) server_negotiation.cc:407] Connection header received
0925 22:05:43.183249 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:43.183250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:43.183259 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:43.183314 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:43.183574 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.184233 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.184964 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.185136 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.185183 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:43.186000 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:43.186001 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:43.186001 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:43.186007 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:43.187462 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:43.187849 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:43.187850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:43.187851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:43.187930 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:43.188106 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:43.188107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:43.188107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:43.188221 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:43.188295 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:43.188460 (+ 165us) server_negotiation.cc:299] Negotiation successful
0925 22:05:43.188540 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 22:05:43.188642 12837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:43.183022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:45363 (local address 127.5.0.65:49845)
0925 22:05:43.183131 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:43.183143 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:43.183196 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:43.183317 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:43.183317 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:43.183462 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:43.183514 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:43.183516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.184327 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.184328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:43.184896 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:43.184898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.185198 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.185198 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:43.185241 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:43.185620 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:43.185624 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:43.185928 (+ 304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:43.187536 (+ 1608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:43.187536 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:43.187537 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:43.187757 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:43.187980 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:43.187981 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:43.187981 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:43.188035 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:43.188298 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:43.188299 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:43.188353 (+ 54us) client_negotiation.cc:769] Sending connection context
0925 22:05:43.188436 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 22:05:43.188537 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":63,"threads_started":1}
WARNING: no policy specified for oryx/127.5.0.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.0.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.0.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.66.keytab.
Entry for principal oryx/127.5.0.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.66.keytab.
I20260925 22:05:43.214336 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.66:0
--local_ip_for_outbound_sockets=127.5.0.66
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:45363
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.66.keytab
--principal=oryx/127.5.0.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:42003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:43.295060 12840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:43.295265 12840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:43.295289 12840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:43.296474 12840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:43.296517 12840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:43.296537 12840 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 22:05:43.296586 12840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.66
I20260925 22:05:43.298067 12840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:42003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.66.keytab
--principal=oryx/127.5.0.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.0.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.0.126:45363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.66
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:43.298287 12840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:43.298491 12840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:43.303510 12846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:43.303524 12840 server_base.cc:1034] running on GCE node
W20260925 22:05:43.303530 12845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:43.303673 12848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:43.303879 12840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:43.304127 12840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:43.305269 12840 hybrid_clock.cc:648] HybridClock initialized: now 1790373943305259 us; error 31 us; skew 500 ppm
Sep 25 22:05:43 dist-test-slave-7mqr krb5kdc[12583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373943, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:43.307713 12840 init.cc:377] Logged in from keytab as oryx/127.5.0.66@KRBTEST.COM (short username oryx)
I20260925 22:05:43.308218 12840 webserver.cc:460] Webserver started at http://127.5.0.66:43229/ using document root <none> and password file <none>
I20260925 22:05:43.308458 12840 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:43.308503 12840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:43.308640 12840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:43.309659 12840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-1/data/instance:
uuid: "bbe00c6771664c5393295b94a89bc4ae"
format_stamp: "Formatted at 2026-09-25 22:05:43 on dist-test-slave-7mqr"
server_key: "bd1ad28968907fccdf8137e29f0dec7e"
server_key_iv: "5b9d1e5dc04233982d85fb1670688d28"
server_key_version: "encryptionkey@0"
I20260925 22:05:43.310015 12840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance:
uuid: "bbe00c6771664c5393295b94a89bc4ae"
format_stamp: "Formatted at 2026-09-25 22:05:43 on dist-test-slave-7mqr"
server_key: "bd1ad28968907fccdf8137e29f0dec7e"
server_key_iv: "5b9d1e5dc04233982d85fb1670688d28"
server_key_version: "encryptionkey@0"
I20260925 22:05:43.311290 12840 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 22:05:43.312032 12855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:43.312222 12840 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:05:43.312311 12840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-1/wal
uuid: "bbe00c6771664c5393295b94a89bc4ae"
format_stamp: "Formatted at 2026-09-25 22:05:43 on dist-test-slave-7mqr"
server_key: "bd1ad28968907fccdf8137e29f0dec7e"
server_key_iv: "5b9d1e5dc04233982d85fb1670688d28"
server_key_version: "encryptionkey@0"
I20260925 22:05:43.312386 12840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:43.342411 12840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:43.344385 12840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:43.344548 12840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:43.345077 12840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:43.345142 12840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:43.345191 12840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:43.345216 12840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 22:05:43 dist-test-slave-7mqr krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373943, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM
I20260925 22:05:43.352648 12840 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.66:40453
I20260925 22:05:43.352741 12970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.66:40453 every 8 connection(s)
I20260925 22:05:43.353085 12840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
I20260925 22:05:43.355232 12677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:43.345640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:57475 (local address 127.5.0.126:45363)
0925 22:05:43.345703 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:43.345705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:43.345897 (+ 192us) server_negotiation.cc:407] Connection header received
0925 22:05:43.346430 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:43.346431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:43.346445 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:43.346510 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:43.347387 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.348151 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.349659 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.349956 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.350037 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:43.352509 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:43.352511 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:43.352511 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:43.352520 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:43.354067 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:43.354429 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:43.354430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:43.354431 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:43.354486 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:43.354742 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:43.354743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:43.354743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:43.354849 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:43.354899 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:43.355100 (+ 201us) server_negotiation.cc:299] Negotiation successful
0925 22:05:43.355171 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 22:05:43.355418 12864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:43.345623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:45363 (local address 127.5.0.66:57475)
0925 22:05:43.345775 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:43.345811 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:43.346324 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:43.346588 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:43.346591 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:43.347009 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:43.347300 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:43.347306 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.348262 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.348263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:43.349556 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:43.349559 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.350054 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.350055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:43.350221 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:43.350798 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:43.350807 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:43.352404 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:43.354137 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:43.354141 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:43.354143 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:43.354356 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:43.354584 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:43.354585 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:43.354585 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:43.354688 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:43.354889 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:43.354892 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:43.354998 (+ 106us) client_negotiation.cc:769] Sending connection context
0925 22:05:43.355073 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 22:05:43.355190 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":79,"threads_started":1}
I20260925 22:05:43.355965 12971 heartbeater.cc:347] Connected to a master server at 127.5.0.126:45363
I20260925 22:05:43.356086 12971 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:43.356267 12971 heartbeater.cc:511] Master 127.5.0.126:45363 requested a full tablet report, sending...
I20260925 22:05:43.356675 12628 ts_manager.cc:194] Registered new tserver with Master: bbe00c6771664c5393295b94a89bc4ae (127.5.0.66:40453)
I20260925 22:05:43.357216 12628 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.0.66@KRBTEST.COM'} at 127.5.0.66:57475
I20260925 22:05:43.359529 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 12840
I20260925 22:05:43.359654 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance
I20260925 22:05:43.359859 5121 external_mini_cluster.cc:1428] Setting key 9730f8a342ba55e6f5ab1dc8b527c654
I20260925 22:05:43.363068 12677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:43.357083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:60049 (local address 127.5.0.126:45363)
0925 22:05:43.357351 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:43.357353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:43.357361 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:05:43.357386 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:43.357386 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:43.357393 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:43.357461 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:43.357816 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.358278 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.358911 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.359076 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.359123 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:43.360192 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:43.360194 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:43.360194 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:43.360201 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:43.361856 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:43.362223 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:43.362224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:43.362225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:43.362283 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:43.362508 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:43.362509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:43.362509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:43.362653 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:43.362732 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:43.362888 (+ 156us) server_negotiation.cc:299] Negotiation successful
0925 22:05:43.362985 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251}
I20260925 22:05:43.363076 12977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:43.357013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:45363 (local address 127.5.0.66:60049)
0925 22:05:43.357135 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:43.357145 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:43.357200 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:43.357467 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:43.357468 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:43.357670 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:43.357738 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:43.357740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.358360 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.358361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:43.358844 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:43.358845 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.359136 (+ 291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.359137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:43.359181 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:43.359659 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:43.359665 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:43.360080 (+ 415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:43.361942 (+ 1862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:43.361943 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:43.361943 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:43.362112 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:43.362358 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:43.362358 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:43.362359 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:43.362426 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:43.362736 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:43.362737 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:43.362779 (+ 42us) client_negotiation.cc:769] Sending connection context
0925 22:05:43.362863 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 22:05:43.362968 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":70,"threads_started":1}
WARNING: no policy specified for oryx/127.5.0.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.0.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.0.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.67.keytab.
Entry for principal oryx/127.5.0.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.67.keytab.
I20260925 22:05:43.386734 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.67:0
--local_ip_for_outbound_sockets=127.5.0.67
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:45363
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.67.keytab
--principal=oryx/127.5.0.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:42003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:43.467350 12980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:43.467584 12980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:43.467622 12980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:43.468961 12980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:43.469029 12980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:43.469063 12980 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 22:05:43.469103 12980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.67
I20260925 22:05:43.470584 12980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:42003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.67.keytab
--principal=oryx/127.5.0.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.0.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.0.126:45363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.67
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:43.470860 12980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:43.471091 12980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:43.475919 12986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:43.475934 12988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:43.475971 12985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:43.476130 12980 server_base.cc:1034] running on GCE node
I20260925 22:05:43.476337 12980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:43.476565 12980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:43.477737 12980 hybrid_clock.cc:648] HybridClock initialized: now 1790373943477725 us; error 38 us; skew 500 ppm
Sep 25 22:05:43 dist-test-slave-7mqr krb5kdc[12583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373943, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:43.480192 12980 init.cc:377] Logged in from keytab as oryx/127.5.0.67@KRBTEST.COM (short username oryx)
I20260925 22:05:43.480649 12980 webserver.cc:460] Webserver started at http://127.5.0.67:43189/ using document root <none> and password file <none>
I20260925 22:05:43.480885 12980 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:43.480950 12980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:43.481101 12980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:43.482168 12980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-2/data/instance:
uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c"
format_stamp: "Formatted at 2026-09-25 22:05:43 on dist-test-slave-7mqr"
server_key: "8cf8902c5a923dcb1bea26d00015fff8"
server_key_iv: "54e030e8da7e40a0778c5db6ffa6d141"
server_key_version: "encryptionkey@0"
I20260925 22:05:43.482527 12980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance:
uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c"
format_stamp: "Formatted at 2026-09-25 22:05:43 on dist-test-slave-7mqr"
server_key: "8cf8902c5a923dcb1bea26d00015fff8"
server_key_iv: "54e030e8da7e40a0778c5db6ffa6d141"
server_key_version: "encryptionkey@0"
I20260925 22:05:43.483798 12980 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 22:05:43.484521 12995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:43.484684 12980 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:05:43.484788 12980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-2/wal
uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c"
format_stamp: "Formatted at 2026-09-25 22:05:43 on dist-test-slave-7mqr"
server_key: "8cf8902c5a923dcb1bea26d00015fff8"
server_key_iv: "54e030e8da7e40a0778c5db6ffa6d141"
server_key_version: "encryptionkey@0"
I20260925 22:05:43.484864 12980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:43.501412 12980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:43.503813 12980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:43.504001 12980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:43.504587 12980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:43.504647 12980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:43.504689 12980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:43.504725 12980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 22:05:43 dist-test-slave-7mqr krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373943, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM
I20260925 22:05:43.512125 12980 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.67:32885
I20260925 22:05:43.512224 13110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.67:32885 every 8 connection(s)
I20260925 22:05:43.512542 12980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
I20260925 22:05:43.514540 12677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:43.504942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:50881 (local address 127.5.0.126:45363)
0925 22:05:43.504998 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:43.505000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:43.505270 (+ 270us) server_negotiation.cc:407] Connection header received
0925 22:05:43.505747 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:43.505748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:43.505762 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:43.505861 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:43.506537 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.507403 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.508718 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.509067 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.509148 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:43.511928 (+ 2780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:43.511930 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:43.511930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:43.511938 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:43.513455 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:43.513811 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:43.513811 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:43.513812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:43.513864 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:43.514063 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:43.514064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:43.514064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:43.514166 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:43.514218 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:43.514423 (+ 205us) server_negotiation.cc:299] Negotiation successful
0925 22:05:43.514479 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 22:05:43.514715 13004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:43.504962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:45363 (local address 127.5.0.67:50881)
0925 22:05:43.505126 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:43.505166 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:43.505636 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:43.505910 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:43.505913 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:43.506200 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:43.506433 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:43.506437 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.507502 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.507503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:43.508618 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:43.508620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.509180 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.509181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:43.509312 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:43.509835 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:43.509846 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:43.511815 (+ 1969us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:43.513528 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:43.513531 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:43.513533 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:43.513743 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:43.513921 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:43.513921 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:43.513922 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:43.514010 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:43.514217 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:43.514221 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:43.514330 (+ 109us) client_negotiation.cc:769] Sending connection context
0925 22:05:43.514409 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 22:05:43.514523 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":98,"threads_started":1}
I20260925 22:05:43.515271 13111 heartbeater.cc:347] Connected to a master server at 127.5.0.126:45363
I20260925 22:05:43.515347 13111 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:43.515517 13111 heartbeater.cc:511] Master 127.5.0.126:45363 requested a full tablet report, sending...
I20260925 22:05:43.515961 12627 ts_manager.cc:194] Registered new tserver with Master: b1d47aae2b464377b14cd3ac5e8f1d2c (127.5.0.67:32885)
I20260925 22:05:43.516490 12627 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.0.67@KRBTEST.COM'} at 127.5.0.67:50881
I20260925 22:05:43.521436 13117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:43.516277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:45363 (local address 127.5.0.67:58617)
0925 22:05:43.516412 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:43.516423 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:43.516481 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:43.516667 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:43.516669 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:43.516863 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:43.516940 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:43.516942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.517545 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.517546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:43.518021 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:43.518022 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.518331 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.518331 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:43.518379 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:43.518749 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:43.518754 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:43.519125 (+ 371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:43.520549 (+ 1424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:43.520550 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:43.520550 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:43.520710 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:43.520879 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:43.520879 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:43.520879 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:43.520928 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:43.521129 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:43.521129 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:43.521160 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 22:05:43.521248 (+ 88us) client_negotiation.cc:240] Negotiation successful
0925 22:05:43.521350 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":86,"threads_started":1}
I20260925 22:05:43.521518 12677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:43.516365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:58617 (local address 127.5.0.126:45363)
0925 22:05:43.516406 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:43.516408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:43.516469 (+ 61us) server_negotiation.cc:407] Connection header received
0925 22:05:43.516536 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:43.516536 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:43.516565 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:43.516620 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:43.517015 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.517460 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.518090 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.518270 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.518318 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:43.519226 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:43.519228 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:43.519228 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:43.519234 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:43.520478 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:43.520775 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:43.520776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:43.520776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:43.520824 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:43.520988 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:43.520989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:43.520989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:43.521078 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:43.521165 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:43.521310 (+ 145us) server_negotiation.cc:299] Negotiation successful
0925 22:05:43.521392 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 22:05:43.521606 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 12980
I20260925 22:05:43.521700 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance
I20260925 22:05:43.521910 5121 external_mini_cluster.cc:1428] Setting key a6d2ba0670b817e131c00cfa2a3fd5d2
I20260925 22:05:43.523008 5121 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 22:05:43 dist-test-slave-7mqr krb5kdc[12583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373943, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 22:05:43 dist-test-slave-7mqr krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:43 dist-test-slave-7mqr krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
W20260925 22:05:43.543748 12677 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:43.538263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35096 (local address 127.5.0.126:45363)
0925 22:05:43.538327 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:43.538329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:43.538339 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:05:43.538388 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:43.538388 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:43.538397 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:43.538457 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:43.539069 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.539558 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.540402 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.540582 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.540637 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:43.543634 (+ 2997us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35096: BlockingRecv error: recv got EOF from 127.0.0.1:35096 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 25 22:05:43 dist-test-slave-7mqr krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373943, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM
I20260925 22:05:43.551106 12677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:43.544664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35098 (local address 127.5.0.126:45363)
0925 22:05:43.544710 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:43.544711 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:43.544786 (+ 75us) server_negotiation.cc:407] Connection header received
0925 22:05:43.544824 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:43.544825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:43.544833 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:43.544881 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:43.545329 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.545798 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.546449 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.546677 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.546744 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:43.548824 (+ 2080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:43.548826 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:43.548826 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:43.548833 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:43.550008 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:43.550328 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:43.550329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:43.550329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:43.550393 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:43.550656 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:43.550656 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:43.550657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:43.550795 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:43.550896 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:43.550978 (+ 82us) server_negotiation.cc:299] Negotiation successful
0925 22:05:43.551040 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 22:05:43.552778 12627 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35098:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:05:43.554363 12627 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:05:43.560909 13137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:43.556919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54710 (local address 127.5.0.67:32885)
0925 22:05:43.557111 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:43.557116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:43.557126 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:05:43.557168 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:43.557171 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:43.557186 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:43.557491 (+ 305us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:43.557551 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.558656 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.559957 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.560446 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.560545 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:43.560623 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:43.560757 (+ 134us) server_negotiation.cc:299] Negotiation successful
0925 22:05:43.560792 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":77,"threads_started":1}
I20260925 22:05:43.560915 13136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:43.556721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:40453 (local address 127.0.0.1:36776)
0925 22:05:43.557025 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:43.557035 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:43.557125 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:43.557439 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:43.557440 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:43.557442 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:43.557484 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:43.557485 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.558765 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.558767 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:43.559688 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:43.559690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.560622 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.560623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:43.560690 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:43.560694 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:05:43.560723 (+ 29us) client_negotiation.cc:240] Negotiation successful
0925 22:05:43.560790 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":2,"thread_start_us":79,"threads_started":1}
I20260925 22:05:43.561117 13138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:43.557120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60000 (local address 127.5.0.65:33097)
0925 22:05:43.557275 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:43.557279 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:43.557409 (+ 130us) server_negotiation.cc:407] Connection header received
0925 22:05:43.557450 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:43.557452 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:43.557466 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:43.557533 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:43.557715 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.558387 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.559815 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.560525 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.560592 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:43.560667 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:43.560959 (+ 292us) server_negotiation.cc:299] Negotiation successful
0925 22:05:43.561007 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":89,"threads_started":1}
I20260925 22:05:43.560923 13133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:43.556911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:33097 (local address 127.0.0.1:60000)
0925 22:05:43.557299 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:43.557309 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:43.557356 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:43.557593 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:43.557594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:43.557596 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:43.557650 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:43.557652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.558474 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.558475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:43.559735 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:43.559738 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.560695 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.560696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:43.560750 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:43.560753 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:05:43.560798 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 22:05:43.560836 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":81,"threads_started":1}
I20260925 22:05:43.561384 13135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:43.556829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36776 (local address 127.5.0.66:40453)
0925 22:05:43.557130 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:43.557135 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:43.557164 (+ 29us) server_negotiation.cc:407] Connection header received
0925 22:05:43.557221 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:43.557223 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:43.557238 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:43.557351 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:43.557565 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.558655 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.559768 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.560520 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.561135 (+ 615us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:43.561218 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:43.561244 (+ 26us) server_negotiation.cc:299] Negotiation successful
0925 22:05:43.561287 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":78,"threads_started":1}
I20260925 22:05:43.560981 13134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:43.556720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:32885 (local address 127.0.0.1:54710)
0925 22:05:43.556885 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:43.556926 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:43.556988 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:43.557297 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:43.557300 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:43.557303 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:43.557398 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:43.557400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.558740 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.558741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:43.559837 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:43.559839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.560546 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.560547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:43.560624 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:43.560639 (+ 15us) client_negotiation.cc:769] Sending connection context
0925 22:05:43.560740 (+ 101us) client_negotiation.cc:240] Negotiation successful
0925 22:05:43.560791 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":50,"threads_started":1}
I20260925 22:05:43.561967 12765 tablet_service.cc:1467] Processing CreateTablet for tablet 82aa7608a9ba4c599d35f63d0afc9a82 (DEFAULT_TABLE table=test-table [id=1df36a2abbd84efd9711b757b874f2fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:43.562044 12905 tablet_service.cc:1467] Processing CreateTablet for tablet 82aa7608a9ba4c599d35f63d0afc9a82 (DEFAULT_TABLE table=test-table [id=1df36a2abbd84efd9711b757b874f2fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:43.562342 12905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82aa7608a9ba4c599d35f63d0afc9a82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:43.562351 12765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82aa7608a9ba4c599d35f63d0afc9a82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:43.565517 13140 tablet_bootstrap.cc:492] T 82aa7608a9ba4c599d35f63d0afc9a82 P 87968f19e8d8461aa549ca61f020e42f: Bootstrap starting.
I20260925 22:05:43.565573 13139 tablet_bootstrap.cc:492] T 82aa7608a9ba4c599d35f63d0afc9a82 P bbe00c6771664c5393295b94a89bc4ae: Bootstrap starting.
I20260925 22:05:43.566138 13139 tablet_bootstrap.cc:654] T 82aa7608a9ba4c599d35f63d0afc9a82 P bbe00c6771664c5393295b94a89bc4ae: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:43.566184 13140 tablet_bootstrap.cc:654] T 82aa7608a9ba4c599d35f63d0afc9a82 P 87968f19e8d8461aa549ca61f020e42f: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:43.566233 13045 tablet_service.cc:1467] Processing CreateTablet for tablet 82aa7608a9ba4c599d35f63d0afc9a82 (DEFAULT_TABLE table=test-table [id=1df36a2abbd84efd9711b757b874f2fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:43.566531 13139 log.cc:824] T 82aa7608a9ba4c599d35f63d0afc9a82 P bbe00c6771664c5393295b94a89bc4ae: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:43.566536 13140 log.cc:824] T 82aa7608a9ba4c599d35f63d0afc9a82 P 87968f19e8d8461aa549ca61f020e42f: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:43.566599 13045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82aa7608a9ba4c599d35f63d0afc9a82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:43.567276 13139 tablet_bootstrap.cc:492] T 82aa7608a9ba4c599d35f63d0afc9a82 P bbe00c6771664c5393295b94a89bc4ae: No bootstrap required, opened a new log
I20260925 22:05:43.567358 13139 ts_tablet_manager.cc:1397] T 82aa7608a9ba4c599d35f63d0afc9a82 P bbe00c6771664c5393295b94a89bc4ae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:05:43.567459 13140 tablet_bootstrap.cc:492] T 82aa7608a9ba4c599d35f63d0afc9a82 P 87968f19e8d8461aa549ca61f020e42f: No bootstrap required, opened a new log
I20260925 22:05:43.567536 13140 ts_tablet_manager.cc:1397] T 82aa7608a9ba4c599d35f63d0afc9a82 P 87968f19e8d8461aa549ca61f020e42f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:05:43.569068 13139 raft_consensus.cc:348] T 82aa7608a9ba4c599d35f63d0afc9a82 P bbe00c6771664c5393295b94a89bc4ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } } peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } }
I20260925 22:05:43.569227 13139 raft_consensus.cc:374] T 82aa7608a9ba4c599d35f63d0afc9a82 P bbe00c6771664c5393295b94a89bc4ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:43.569272 13139 raft_consensus.cc:729] T 82aa7608a9ba4c599d35f63d0afc9a82 P bbe00c6771664c5393295b94a89bc4ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbe00c6771664c5393295b94a89bc4ae, State: Initialized, Role: FOLLOWER
I20260925 22:05:43.569416 13139 consensus_queue.cc:260] T 82aa7608a9ba4c599d35f63d0afc9a82 P bbe00c6771664c5393295b94a89bc4ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } } peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } }
I20260925 22:05:43.569705 13139 ts_tablet_manager.cc:1428] T 82aa7608a9ba4c599d35f63d0afc9a82 P bbe00c6771664c5393295b94a89bc4ae: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260925 22:05:43.569718 12971 heartbeater.cc:503] Master 127.5.0.126:45363 was elected leader, sending a full tablet report...
I20260925 22:05:43.569778 13140 raft_consensus.cc:348] T 82aa7608a9ba4c599d35f63d0afc9a82 P 87968f19e8d8461aa549ca61f020e42f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } } peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } }
I20260925 22:05:43.569931 13140 raft_consensus.cc:374] T 82aa7608a9ba4c599d35f63d0afc9a82 P 87968f19e8d8461aa549ca61f020e42f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:43.569962 13140 raft_consensus.cc:729] T 82aa7608a9ba4c599d35f63d0afc9a82 P 87968f19e8d8461aa549ca61f020e42f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87968f19e8d8461aa549ca61f020e42f, State: Initialized, Role: FOLLOWER
I20260925 22:05:43.570103 13140 consensus_queue.cc:260] T 82aa7608a9ba4c599d35f63d0afc9a82 P 87968f19e8d8461aa549ca61f020e42f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } } peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } }
I20260925 22:05:43.570246 13144 tablet_bootstrap.cc:492] T 82aa7608a9ba4c599d35f63d0afc9a82 P b1d47aae2b464377b14cd3ac5e8f1d2c: Bootstrap starting.
I20260925 22:05:43.570376 13140 ts_tablet_manager.cc:1428] T 82aa7608a9ba4c599d35f63d0afc9a82 P 87968f19e8d8461aa549ca61f020e42f: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260925 22:05:43.570654 12831 heartbeater.cc:503] Master 127.5.0.126:45363 was elected leader, sending a full tablet report...
I20260925 22:05:43.570943 13144 tablet_bootstrap.cc:654] T 82aa7608a9ba4c599d35f63d0afc9a82 P b1d47aae2b464377b14cd3ac5e8f1d2c: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:43.571275 13144 log.cc:824] T 82aa7608a9ba4c599d35f63d0afc9a82 P b1d47aae2b464377b14cd3ac5e8f1d2c: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:43.572079 13144 tablet_bootstrap.cc:492] T 82aa7608a9ba4c599d35f63d0afc9a82 P b1d47aae2b464377b14cd3ac5e8f1d2c: No bootstrap required, opened a new log
I20260925 22:05:43.572156 13144 ts_tablet_manager.cc:1397] T 82aa7608a9ba4c599d35f63d0afc9a82 P b1d47aae2b464377b14cd3ac5e8f1d2c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:05:43.573596 13144 raft_consensus.cc:348] T 82aa7608a9ba4c599d35f63d0afc9a82 P b1d47aae2b464377b14cd3ac5e8f1d2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } } peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } }
I20260925 22:05:43.573745 13144 raft_consensus.cc:374] T 82aa7608a9ba4c599d35f63d0afc9a82 P b1d47aae2b464377b14cd3ac5e8f1d2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:43.573783 13144 raft_consensus.cc:729] T 82aa7608a9ba4c599d35f63d0afc9a82 P b1d47aae2b464377b14cd3ac5e8f1d2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1d47aae2b464377b14cd3ac5e8f1d2c, State: Initialized, Role: FOLLOWER
I20260925 22:05:43.573892 13144 consensus_queue.cc:260] T 82aa7608a9ba4c599d35f63d0afc9a82 P b1d47aae2b464377b14cd3ac5e8f1d2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } } peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } }
I20260925 22:05:43.574137 13144 ts_tablet_manager.cc:1428] T 82aa7608a9ba4c599d35f63d0afc9a82 P b1d47aae2b464377b14cd3ac5e8f1d2c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:05:43.574225 13111 heartbeater.cc:503] Master 127.5.0.126:45363 was elected leader, sending a full tablet report...
W20260925 22:05:43.603645 12972 tablet.cc:2369] T 82aa7608a9ba4c599d35f63d0afc9a82 P bbe00c6771664c5393295b94a89bc4ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 22:05:43.678758 12832 tablet.cc:2369] T 82aa7608a9ba4c599d35f63d0afc9a82 P 87968f19e8d8461aa549ca61f020e42f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 22:05:43.763164 13112 tablet.cc:2369] T 82aa7608a9ba4c599d35f63d0afc9a82 P b1d47aae2b464377b14cd3ac5e8f1d2c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:05:43.774668 13147 raft_consensus.cc:482] T 82aa7608a9ba4c599d35f63d0afc9a82 P b1d47aae2b464377b14cd3ac5e8f1d2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:05:43.774792 13147 raft_consensus.cc:504] T 82aa7608a9ba4c599d35f63d0afc9a82 P b1d47aae2b464377b14cd3ac5e8f1d2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } } peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } }
I20260925 22:05:43.775115 13147 leader_election.cc:288] T 82aa7608a9ba4c599d35f63d0afc9a82 P b1d47aae2b464377b14cd3ac5e8f1d2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bbe00c6771664c5393295b94a89bc4ae (127.5.0.66:40453), 87968f19e8d8461aa549ca61f020e42f (127.5.0.65:33097)
I20260925 22:05:43.778169 13135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:43.775366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:40641 (local address 127.5.0.66:40453)
0925 22:05:43.775413 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:43.775415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:43.775644 (+ 229us) server_negotiation.cc:407] Connection header received
0925 22:05:43.775726 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:43.775727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:43.775737 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:43.775808 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:43.775994 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.776532 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.777474 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.777828 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.777912 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:43.777922 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:43.778060 (+ 138us) server_negotiation.cc:299] Negotiation successful
0925 22:05:43.778096 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 22:05:43.778182 13138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:43.775378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:56945 (local address 127.5.0.65:33097)
0925 22:05:43.775420 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:43.775422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:43.775431 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:05:43.775500 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:43.775501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:43.775517 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:43.775571 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:43.775734 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.776334 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.777420 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.777827 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.777912 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:43.777922 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:43.778080 (+ 158us) server_negotiation.cc:299] Negotiation successful
0925 22:05:43.778113 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 22:05:43.778178 13004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:43.775330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:33097 (local address 127.5.0.67:56945)
0925 22:05:43.775371 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:43.775383 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:43.775431 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:43.775589 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:43.775589 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:43.775591 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:43.775661 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:43.775663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.776423 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.776424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:43.777340 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:43.777342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.777913 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.777914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:43.777995 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:43.777998 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:05:43.778068 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 22:05:43.778106 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19,"mutex_wait_us":21}
I20260925 22:05:43.778192 13148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:43.775337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:40453 (local address 127.5.0.67:40641)
0925 22:05:43.775572 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:43.775581 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:43.775656 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:43.775811 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:43.775812 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:43.775813 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:43.775928 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:43.775930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.776614 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.776614 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:43.777374 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:43.777390 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.777911 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.777912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:43.777991 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:43.777995 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:05:43.778037 (+ 42us) client_negotiation.cc:240] Negotiation successful
0925 22:05:43.778094 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":39,"thread_start_us":135,"threads_started":1}
I20260925 22:05:43.778568 12785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82aa7608a9ba4c599d35f63d0afc9a82" candidate_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87968f19e8d8461aa549ca61f020e42f" is_pre_election: true
I20260925 22:05:43.778705 12785 raft_consensus.cc:2454] T 82aa7608a9ba4c599d35f63d0afc9a82 P 87968f19e8d8461aa549ca61f020e42f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1d47aae2b464377b14cd3ac5e8f1d2c in term 0.
I20260925 22:05:43.778934 12997 leader_election.cc:302] T 82aa7608a9ba4c599d35f63d0afc9a82 P b1d47aae2b464377b14cd3ac5e8f1d2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87968f19e8d8461aa549ca61f020e42f, b1d47aae2b464377b14cd3ac5e8f1d2c; no voters:
I20260925 22:05:43.779093 13147 raft_consensus.cc:2781] T 82aa7608a9ba4c599d35f63d0afc9a82 P b1d47aae2b464377b14cd3ac5e8f1d2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:05:43.779085 12908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82aa7608a9ba4c599d35f63d0afc9a82" candidate_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbe00c6771664c5393295b94a89bc4ae" is_pre_election: true
I20260925 22:05:43.779161 13147 raft_consensus.cc:482] T 82aa7608a9ba4c599d35f63d0afc9a82 P b1d47aae2b464377b14cd3ac5e8f1d2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:05:43.779206 13147 raft_consensus.cc:3037] T 82aa7608a9ba4c599d35f63d0afc9a82 P b1d47aae2b464377b14cd3ac5e8f1d2c [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:43.779248 12908 raft_consensus.cc:2454] T 82aa7608a9ba4c599d35f63d0afc9a82 P bbe00c6771664c5393295b94a89bc4ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1d47aae2b464377b14cd3ac5e8f1d2c in term 0.
I20260925 22:05:43.780265 13147 raft_consensus.cc:504] T 82aa7608a9ba4c599d35f63d0afc9a82 P b1d47aae2b464377b14cd3ac5e8f1d2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } } peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } }
I20260925 22:05:43.780459 13147 leader_election.cc:288] T 82aa7608a9ba4c599d35f63d0afc9a82 P b1d47aae2b464377b14cd3ac5e8f1d2c [CANDIDATE]: Term 1 election: Requested vote from peers bbe00c6771664c5393295b94a89bc4ae (127.5.0.66:40453), 87968f19e8d8461aa549ca61f020e42f (127.5.0.65:33097)
I20260925 22:05:43.780589 12908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82aa7608a9ba4c599d35f63d0afc9a82" candidate_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbe00c6771664c5393295b94a89bc4ae"
I20260925 22:05:43.780628 12785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82aa7608a9ba4c599d35f63d0afc9a82" candidate_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87968f19e8d8461aa549ca61f020e42f"
I20260925 22:05:43.780668 12908 raft_consensus.cc:3037] T 82aa7608a9ba4c599d35f63d0afc9a82 P bbe00c6771664c5393295b94a89bc4ae [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:43.780715 12785 raft_consensus.cc:3037] T 82aa7608a9ba4c599d35f63d0afc9a82 P 87968f19e8d8461aa549ca61f020e42f [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:43.781342 12908 raft_consensus.cc:2454] T 82aa7608a9ba4c599d35f63d0afc9a82 P bbe00c6771664c5393295b94a89bc4ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1d47aae2b464377b14cd3ac5e8f1d2c in term 1.
I20260925 22:05:43.781420 12785 raft_consensus.cc:2454] T 82aa7608a9ba4c599d35f63d0afc9a82 P 87968f19e8d8461aa549ca61f020e42f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1d47aae2b464377b14cd3ac5e8f1d2c in term 1.
I20260925 22:05:43.781527 12998 leader_election.cc:302] T 82aa7608a9ba4c599d35f63d0afc9a82 P b1d47aae2b464377b14cd3ac5e8f1d2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1d47aae2b464377b14cd3ac5e8f1d2c, bbe00c6771664c5393295b94a89bc4ae; no voters:
I20260925 22:05:43.781615 13147 raft_consensus.cc:2781] T 82aa7608a9ba4c599d35f63d0afc9a82 P b1d47aae2b464377b14cd3ac5e8f1d2c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:43.781751 13147 raft_consensus.cc:686] T 82aa7608a9ba4c599d35f63d0afc9a82 P b1d47aae2b464377b14cd3ac5e8f1d2c [term 1 LEADER]: Becoming Leader. State: Replica: b1d47aae2b464377b14cd3ac5e8f1d2c, State: Running, Role: LEADER
I20260925 22:05:43.781872 13147 consensus_queue.cc:237] T 82aa7608a9ba4c599d35f63d0afc9a82 P b1d47aae2b464377b14cd3ac5e8f1d2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } } peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } }
I20260925 22:05:43.782611 12627 catalog_manager.cc:5495] T 82aa7608a9ba4c599d35f63d0afc9a82 P b1d47aae2b464377b14cd3ac5e8f1d2c reported cstate change: term changed from 0 to 1, leader changed from <none> to b1d47aae2b464377b14cd3ac5e8f1d2c (127.5.0.67). New cstate: current_term: 1 leader_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } health_report { overall_health: UNKNOWN } } }
I20260925 22:05:43.842796 13137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:43.840450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54716 (local address 127.5.0.67:32885)
0925 22:05:43.840513 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:43.840515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:43.840525 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:05:43.840569 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:43.840569 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:43.840589 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:43.840663 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:05:43.840807 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.841268 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.842020 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:43.842213 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:43.842287 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:43.842468 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:05:43.842563 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:05:43.842645 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:43.842685 (+ 40us) server_negotiation.cc:299] Negotiation successful
0925 22:05:43.842706 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 22:05:43.844682 12785 raft_consensus.cc:1261] T 82aa7608a9ba4c599d35f63d0afc9a82 P 87968f19e8d8461aa549ca61f020e42f [term 1 FOLLOWER]: Refusing update from remote peer b1d47aae2b464377b14cd3ac5e8f1d2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:05:43.844682 12908 raft_consensus.cc:1261] T 82aa7608a9ba4c599d35f63d0afc9a82 P bbe00c6771664c5393295b94a89bc4ae [term 1 FOLLOWER]: Refusing update from remote peer b1d47aae2b464377b14cd3ac5e8f1d2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:05:43.845054 13149 consensus_queue.cc:1035] T 82aa7608a9ba4c599d35f63d0afc9a82 P b1d47aae2b464377b14cd3ac5e8f1d2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:05:43.845209 13147 consensus_queue.cc:1035] T 82aa7608a9ba4c599d35f63d0afc9a82 P b1d47aae2b464377b14cd3ac5e8f1d2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:05:43.847369 13152 mvcc.cc:205] Tried to move back new op lower bound from 7333371673984827392 to 7333371673731026944. Current Snapshot: MvccSnapshot[applied={T|T < 7333371673984827392}]
I20260925 22:05:43.847978 13154 mvcc.cc:205] Tried to move back new op lower bound from 7333371673984827392 to 7333371673731026944. Current Snapshot: MvccSnapshot[applied={T|T < 7333371673984827392}]
I20260925 22:05:43.849766 12627 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:05:43.849861 12627 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:05:43.850613 12627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f5a748e94d32429ca130f24868a54d8c: Sending DeleteTablet for 3 replicas of tablet 82aa7608a9ba4c599d35f63d0afc9a82
I20260925 22:05:43.851037 12765 tablet_service.cc:1514] Processing DeleteTablet for tablet 82aa7608a9ba4c599d35f63d0afc9a82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:43 UTC) from {username='oryx', principal='oryx/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:60000
I20260925 22:05:43.851020 12905 tablet_service.cc:1514] Processing DeleteTablet for tablet 82aa7608a9ba4c599d35f63d0afc9a82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:43 UTC) from {username='oryx', principal='oryx/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:36776
I20260925 22:05:43.850965 13045 tablet_service.cc:1514] Processing DeleteTablet for tablet 82aa7608a9ba4c599d35f63d0afc9a82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:43 UTC) from {username='oryx', principal='oryx/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:54710
I20260925 22:05:43.851348 12627 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35098:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:05:43.851490 12627 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:05:43.851518 13163 tablet_replica.cc:330] T 82aa7608a9ba4c599d35f63d0afc9a82 P bbe00c6771664c5393295b94a89bc4ae: stopping tablet replica
I20260925 22:05:43.851621 13163 raft_consensus.cc:2229] T 82aa7608a9ba4c599d35f63d0afc9a82 P bbe00c6771664c5393295b94a89bc4ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:05:43.851754 13163 raft_consensus.cc:2258] T 82aa7608a9ba4c599d35f63d0afc9a82 P bbe00c6771664c5393295b94a89bc4ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:43.851275 13162 tablet_replica.cc:330] T 82aa7608a9ba4c599d35f63d0afc9a82 P 87968f19e8d8461aa549ca61f020e42f: stopping tablet replica
I20260925 22:05:43.851909 13162 raft_consensus.cc:2229] T 82aa7608a9ba4c599d35f63d0afc9a82 P 87968f19e8d8461aa549ca61f020e42f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:05:43.852427 13162 raft_consensus.cc:2258] T 82aa7608a9ba4c599d35f63d0afc9a82 P 87968f19e8d8461aa549ca61f020e42f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:43.852622 13163 ts_tablet_manager.cc:1905] T 82aa7608a9ba4c599d35f63d0afc9a82 P bbe00c6771664c5393295b94a89bc4ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:43.852834 12765 tablet_service.cc:1467] Processing CreateTablet for tablet 3dd9b6f5b6ac40a1a1a41efa43c2674b (DEFAULT_TABLE table=test-table [id=c6281e95f2e7463591552b0eb395ed6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:43.852833 13045 tablet_service.cc:1467] Processing CreateTablet for tablet 3dd9b6f5b6ac40a1a1a41efa43c2674b (DEFAULT_TABLE table=test-table [id=c6281e95f2e7463591552b0eb395ed6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:43.852845 12905 tablet_service.cc:1467] Processing CreateTablet for tablet 3dd9b6f5b6ac40a1a1a41efa43c2674b (DEFAULT_TABLE table=test-table [id=c6281e95f2e7463591552b0eb395ed6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:43.852952 12765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3dd9b6f5b6ac40a1a1a41efa43c2674b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:43.852950 13045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3dd9b6f5b6ac40a1a1a41efa43c2674b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:43.852950 12905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3dd9b6f5b6ac40a1a1a41efa43c2674b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:43.853911 13140 tablet_bootstrap.cc:492] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P 87968f19e8d8461aa549ca61f020e42f: Bootstrap starting.
I20260925 22:05:43.854274 13140 tablet_bootstrap.cc:654] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P 87968f19e8d8461aa549ca61f020e42f: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:43.854298 13139 tablet_bootstrap.cc:492] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P bbe00c6771664c5393295b94a89bc4ae: Bootstrap starting.
I20260925 22:05:43.854576 13162 ts_tablet_manager.cc:1905] T 82aa7608a9ba4c599d35f63d0afc9a82 P 87968f19e8d8461aa549ca61f020e42f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:43.854681 13139 tablet_bootstrap.cc:654] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P bbe00c6771664c5393295b94a89bc4ae: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:43.854984 13144 tablet_bootstrap.cc:492] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P b1d47aae2b464377b14cd3ac5e8f1d2c: Bootstrap starting.
I20260925 22:05:43.855386 13144 tablet_bootstrap.cc:654] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P b1d47aae2b464377b14cd3ac5e8f1d2c: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:43.856098 13162 ts_tablet_manager.cc:1918] T 82aa7608a9ba4c599d35f63d0afc9a82 P 87968f19e8d8461aa549ca61f020e42f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:05:43.856169 13162 log.cc:1196] T 82aa7608a9ba4c599d35f63d0afc9a82 P 87968f19e8d8461aa549ca61f020e42f: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-0/wal/wals/82aa7608a9ba4c599d35f63d0afc9a82
I20260925 22:05:43.856240 13139 tablet_bootstrap.cc:492] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P bbe00c6771664c5393295b94a89bc4ae: No bootstrap required, opened a new log
I20260925 22:05:43.856285 13139 ts_tablet_manager.cc:1397] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P bbe00c6771664c5393295b94a89bc4ae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 22:05:43.856348 13163 ts_tablet_manager.cc:1918] T 82aa7608a9ba4c599d35f63d0afc9a82 P bbe00c6771664c5393295b94a89bc4ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:05:43.856398 13163 log.cc:1196] T 82aa7608a9ba4c599d35f63d0afc9a82 P bbe00c6771664c5393295b94a89bc4ae: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-1/wal/wals/82aa7608a9ba4c599d35f63d0afc9a82
I20260925 22:05:43.856422 13162 ts_tablet_manager.cc:1939] T 82aa7608a9ba4c599d35f63d0afc9a82 P 87968f19e8d8461aa549ca61f020e42f: Deleting consensus metadata
I20260925 22:05:43.856417 13139 raft_consensus.cc:348] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P bbe00c6771664c5393295b94a89bc4ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:43.856485 13139 raft_consensus.cc:374] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P bbe00c6771664c5393295b94a89bc4ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:43.856511 13139 raft_consensus.cc:729] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P bbe00c6771664c5393295b94a89bc4ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbe00c6771664c5393295b94a89bc4ae, State: Initialized, Role: FOLLOWER
I20260925 22:05:43.856561 13139 consensus_queue.cc:260] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P bbe00c6771664c5393295b94a89bc4ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:43.856650 13139 ts_tablet_manager.cc:1428] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P bbe00c6771664c5393295b94a89bc4ae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:43.856841 12613 catalog_manager.cc:4841] TS 87968f19e8d8461aa549ca61f020e42f (127.5.0.65:33097): tablet 82aa7608a9ba4c599d35f63d0afc9a82 (table test-table [id=1df36a2abbd84efd9711b757b874f2fd]) successfully deleted
I20260925 22:05:43.856896 13164 tablet_replica.cc:330] T 82aa7608a9ba4c599d35f63d0afc9a82 P b1d47aae2b464377b14cd3ac5e8f1d2c: stopping tablet replica
I20260925 22:05:43.856997 13164 raft_consensus.cc:2229] T 82aa7608a9ba4c599d35f63d0afc9a82 P b1d47aae2b464377b14cd3ac5e8f1d2c [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:05:43.857121 13164 raft_consensus.cc:2258] T 82aa7608a9ba4c599d35f63d0afc9a82 P b1d47aae2b464377b14cd3ac5e8f1d2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:43.857219 13144 tablet_bootstrap.cc:492] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P b1d47aae2b464377b14cd3ac5e8f1d2c: No bootstrap required, opened a new log
I20260925 22:05:43.857259 13144 ts_tablet_manager.cc:1397] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P b1d47aae2b464377b14cd3ac5e8f1d2c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:05:43.857393 13144 raft_consensus.cc:348] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P b1d47aae2b464377b14cd3ac5e8f1d2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:43.857458 13144 raft_consensus.cc:374] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P b1d47aae2b464377b14cd3ac5e8f1d2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:43.857479 13144 raft_consensus.cc:729] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P b1d47aae2b464377b14cd3ac5e8f1d2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1d47aae2b464377b14cd3ac5e8f1d2c, State: Initialized, Role: FOLLOWER
I20260925 22:05:43.857518 13144 consensus_queue.cc:260] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P b1d47aae2b464377b14cd3ac5e8f1d2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:43.857529 13163 ts_tablet_manager.cc:1939] T 82aa7608a9ba4c599d35f63d0afc9a82 P bbe00c6771664c5393295b94a89bc4ae: Deleting consensus metadata
I20260925 22:05:43.857596 13144 ts_tablet_manager.cc:1428] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P b1d47aae2b464377b14cd3ac5e8f1d2c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:43.857929 12614 catalog_manager.cc:4841] TS bbe00c6771664c5393295b94a89bc4ae (127.5.0.66:40453): tablet 82aa7608a9ba4c599d35f63d0afc9a82 (table test-table [id=1df36a2abbd84efd9711b757b874f2fd]) successfully deleted
I20260925 22:05:43.864213 13164 ts_tablet_manager.cc:1905] T 82aa7608a9ba4c599d35f63d0afc9a82 P b1d47aae2b464377b14cd3ac5e8f1d2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:43.864307 13140 tablet_bootstrap.cc:492] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P 87968f19e8d8461aa549ca61f020e42f: No bootstrap required, opened a new log
I20260925 22:05:43.864359 13140 ts_tablet_manager.cc:1397] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P 87968f19e8d8461aa549ca61f020e42f: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.001s
I20260925 22:05:43.864490 13140 raft_consensus.cc:348] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P 87968f19e8d8461aa549ca61f020e42f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:43.864560 13140 raft_consensus.cc:374] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P 87968f19e8d8461aa549ca61f020e42f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:43.864580 13140 raft_consensus.cc:729] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P 87968f19e8d8461aa549ca61f020e42f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87968f19e8d8461aa549ca61f020e42f, State: Initialized, Role: FOLLOWER
I20260925 22:05:43.864626 13140 consensus_queue.cc:260] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P 87968f19e8d8461aa549ca61f020e42f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:43.864724 13140 ts_tablet_manager.cc:1428] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P 87968f19e8d8461aa549ca61f020e42f: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260925 22:05:43.865701 13164 ts_tablet_manager.cc:1918] T 82aa7608a9ba4c599d35f63d0afc9a82 P b1d47aae2b464377b14cd3ac5e8f1d2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:05:43.865772 13164 log.cc:1196] T 82aa7608a9ba4c599d35f63d0afc9a82 P b1d47aae2b464377b14cd3ac5e8f1d2c: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-2/wal/wals/82aa7608a9ba4c599d35f63d0afc9a82
I20260925 22:05:43.866032 13164 ts_tablet_manager.cc:1939] T 82aa7608a9ba4c599d35f63d0afc9a82 P b1d47aae2b464377b14cd3ac5e8f1d2c: Deleting consensus metadata
I20260925 22:05:43.866406 12613 catalog_manager.cc:4841] TS b1d47aae2b464377b14cd3ac5e8f1d2c (127.5.0.67:32885): tablet 82aa7608a9ba4c599d35f63d0afc9a82 (table test-table [id=1df36a2abbd84efd9711b757b874f2fd]) successfully deleted
I20260925 22:05:44.214215 13143 raft_consensus.cc:482] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P bbe00c6771664c5393295b94a89bc4ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:05:44.214313 13143 raft_consensus.cc:504] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P bbe00c6771664c5393295b94a89bc4ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:44.214629 13143 leader_election.cc:288] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P bbe00c6771664c5393295b94a89bc4ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87968f19e8d8461aa549ca61f020e42f (127.5.0.65:33097), b1d47aae2b464377b14cd3ac5e8f1d2c (127.5.0.67:32885)
I20260925 22:05:44.217875 13138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:44.214930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:60897 (local address 127.5.0.65:33097)
0925 22:05:44.214973 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:44.214976 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:44.215043 (+ 67us) server_negotiation.cc:407] Connection header received
0925 22:05:44.215135 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:44.215136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:44.215145 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:44.215194 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:44.215409 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.216099 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.217170 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.217524 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.217603 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:44.217614 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:44.217767 (+ 153us) server_negotiation.cc:299] Negotiation successful
0925 22:05:44.217802 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 22:05:44.217878 13137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:44.215102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:34611 (local address 127.5.0.67:32885)
0925 22:05:44.215158 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:44.215160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:44.215171 (+ 11us) server_negotiation.cc:407] Connection header received
0925 22:05:44.215312 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:44.215313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:44.215328 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:44.215392 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:44.215542 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.216063 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.217170 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.217525 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.217607 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:44.217617 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:44.217773 (+ 156us) server_negotiation.cc:299] Negotiation successful
0925 22:05:44.217807 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 22:05:44.217895 13168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:44.214845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:33097 (local address 127.5.0.66:60897)
0925 22:05:44.214965 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:44.214976 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:44.215043 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:44.215252 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:44.215254 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:44.215257 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:44.215337 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:44.215338 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.216187 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.216189 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:44.217070 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:44.217072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.217603 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.217604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:44.217693 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:44.217697 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:05:44.217763 (+ 66us) client_negotiation.cc:240] Negotiation successful
0925 22:05:44.217803 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":1,"thread_start_us":71,"threads_started":1}
I20260925 22:05:44.217901 13169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:44.215027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:32885 (local address 127.5.0.66:34611)
0925 22:05:44.215130 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:44.215140 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:44.215180 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:44.215398 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:44.215399 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:44.215401 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:44.215471 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:44.215473 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.216176 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.216177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:44.217075 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:44.217077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.217609 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.217610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:44.217697 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:44.217702 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:05:44.217769 (+ 67us) client_negotiation.cc:240] Negotiation successful
0925 22:05:44.217811 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":65,"thread_start_us":46,"threads_started":1}
I20260925 22:05:44.218106 12785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dd9b6f5b6ac40a1a1a41efa43c2674b" candidate_uuid: "bbe00c6771664c5393295b94a89bc4ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87968f19e8d8461aa549ca61f020e42f" is_pre_election: true
I20260925 22:05:44.218182 12785 raft_consensus.cc:2454] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P 87968f19e8d8461aa549ca61f020e42f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbe00c6771664c5393295b94a89bc4ae in term 0.
I20260925 22:05:44.218304 13065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dd9b6f5b6ac40a1a1a41efa43c2674b" candidate_uuid: "bbe00c6771664c5393295b94a89bc4ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" is_pre_election: true
I20260925 22:05:44.218349 12857 leader_election.cc:302] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P bbe00c6771664c5393295b94a89bc4ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87968f19e8d8461aa549ca61f020e42f, bbe00c6771664c5393295b94a89bc4ae; no voters:
I20260925 22:05:44.218456 13065 raft_consensus.cc:2454] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P b1d47aae2b464377b14cd3ac5e8f1d2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbe00c6771664c5393295b94a89bc4ae in term 0.
I20260925 22:05:44.218490 13143 raft_consensus.cc:2781] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P bbe00c6771664c5393295b94a89bc4ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:05:44.218524 13143 raft_consensus.cc:482] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P bbe00c6771664c5393295b94a89bc4ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:05:44.218557 13143 raft_consensus.cc:3037] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P bbe00c6771664c5393295b94a89bc4ae [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:44.219221 13143 raft_consensus.cc:504] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P bbe00c6771664c5393295b94a89bc4ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:44.219347 13143 leader_election.cc:288] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P bbe00c6771664c5393295b94a89bc4ae [CANDIDATE]: Term 1 election: Requested vote from peers 87968f19e8d8461aa549ca61f020e42f (127.5.0.65:33097), b1d47aae2b464377b14cd3ac5e8f1d2c (127.5.0.67:32885)
I20260925 22:05:44.219504 12785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dd9b6f5b6ac40a1a1a41efa43c2674b" candidate_uuid: "bbe00c6771664c5393295b94a89bc4ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87968f19e8d8461aa549ca61f020e42f"
I20260925 22:05:44.219539 13065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dd9b6f5b6ac40a1a1a41efa43c2674b" candidate_uuid: "bbe00c6771664c5393295b94a89bc4ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c"
I20260925 22:05:44.219592 13065 raft_consensus.cc:3037] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P b1d47aae2b464377b14cd3ac5e8f1d2c [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:44.219589 12785 raft_consensus.cc:3037] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P 87968f19e8d8461aa549ca61f020e42f [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:44.220253 12785 raft_consensus.cc:2454] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P 87968f19e8d8461aa549ca61f020e42f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbe00c6771664c5393295b94a89bc4ae in term 1.
I20260925 22:05:44.220275 13065 raft_consensus.cc:2454] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P b1d47aae2b464377b14cd3ac5e8f1d2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbe00c6771664c5393295b94a89bc4ae in term 1.
I20260925 22:05:44.220448 12857 leader_election.cc:302] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P bbe00c6771664c5393295b94a89bc4ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1d47aae2b464377b14cd3ac5e8f1d2c, bbe00c6771664c5393295b94a89bc4ae; no voters:
I20260925 22:05:44.220585 13143 raft_consensus.cc:2781] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P bbe00c6771664c5393295b94a89bc4ae [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:44.220712 13143 raft_consensus.cc:686] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P bbe00c6771664c5393295b94a89bc4ae [term 1 LEADER]: Becoming Leader. State: Replica: bbe00c6771664c5393295b94a89bc4ae, State: Running, Role: LEADER
I20260925 22:05:44.220788 13143 consensus_queue.cc:237] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P bbe00c6771664c5393295b94a89bc4ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:44.221513 12627 catalog_manager.cc:5495] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P bbe00c6771664c5393295b94a89bc4ae reported cstate change: term changed from 0 to 1, leader changed from <none> to bbe00c6771664c5393295b94a89bc4ae (127.5.0.66). New cstate: current_term: 1 leader_uuid: "bbe00c6771664c5393295b94a89bc4ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } health_report { overall_health: UNKNOWN } } }
Sep 25 22:05:44 dist-test-slave-7mqr krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373942, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.126@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM
I20260925 22:05:44.319674 13177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:44.312692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:45363 (local address 127.0.0.1:35102)
0925 22:05:44.312860 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:44.312888 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:44.312964 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:44.313148 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:44.313149 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:44.313471 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:44.313521 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:44.313523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.314214 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.314216 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:44.314798 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:44.314800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.315211 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.315212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:44.315266 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:44.315797 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:44.315802 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:44.317336 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:44.318752 (+ 1416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:44.318758 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:44.318760 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:44.318956 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:44.319126 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:44.319126 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:44.319126 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:44.319180 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:44.319391 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:44.319396 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:44.319437 (+ 41us) client_negotiation.cc:769] Sending connection context
0925 22:05:44.319489 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 22:05:44.319581 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":103,"threads_started":1}
I20260925 22:05:44.319754 13178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:44.312794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35102 (local address 127.5.0.126:45363)
0925 22:05:44.312933 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:44.312935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:44.312946 (+ 11us) server_negotiation.cc:407] Connection header received
0925 22:05:44.313040 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:44.313041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:44.313052 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:44.313133 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:44.313613 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.314117 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.314890 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.315118 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.315184 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:44.317412 (+ 2228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:44.317414 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:44.317414 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:44.317420 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:44.318684 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:44.319018 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:44.319019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:44.319019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:44.319070 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:44.319241 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:44.319241 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:44.319241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:44.319338 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:44.319386 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:44.319520 (+ 134us) server_negotiation.cc:299] Negotiation successful
0925 22:05:44.319627 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":116,"thread_start_us":80,"threads_started":1}
I20260925 22:05:44.321563 12627 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:35102:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 22:05:44.321754 12627 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:05:44.323091 12765 tablet_service.cc:1467] Processing CreateTablet for tablet 2fb3c3efe2374e68bff8a31348ffdfd3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7fabdefa52dc4515ad2929657f2252ac]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 22:05:44.323031 12905 tablet_service.cc:1467] Processing CreateTablet for tablet 2fb3c3efe2374e68bff8a31348ffdfd3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7fabdefa52dc4515ad2929657f2252ac]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 22:05:44.323161 13045 tablet_service.cc:1467] Processing CreateTablet for tablet 2fb3c3efe2374e68bff8a31348ffdfd3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7fabdefa52dc4515ad2929657f2252ac]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 22:05:44.323272 12765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fb3c3efe2374e68bff8a31348ffdfd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:44.323288 12905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fb3c3efe2374e68bff8a31348ffdfd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:44.323297 13045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fb3c3efe2374e68bff8a31348ffdfd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:44.324577 13140 tablet_bootstrap.cc:492] T 2fb3c3efe2374e68bff8a31348ffdfd3 P 87968f19e8d8461aa549ca61f020e42f: Bootstrap starting.
I20260925 22:05:44.324765 13139 tablet_bootstrap.cc:492] T 2fb3c3efe2374e68bff8a31348ffdfd3 P bbe00c6771664c5393295b94a89bc4ae: Bootstrap starting.
I20260925 22:05:44.324779 13144 tablet_bootstrap.cc:492] T 2fb3c3efe2374e68bff8a31348ffdfd3 P b1d47aae2b464377b14cd3ac5e8f1d2c: Bootstrap starting.
I20260925 22:05:44.325011 13140 tablet_bootstrap.cc:654] T 2fb3c3efe2374e68bff8a31348ffdfd3 P 87968f19e8d8461aa549ca61f020e42f: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:44.325091 13144 tablet_bootstrap.cc:654] T 2fb3c3efe2374e68bff8a31348ffdfd3 P b1d47aae2b464377b14cd3ac5e8f1d2c: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:44.325158 13139 tablet_bootstrap.cc:654] T 2fb3c3efe2374e68bff8a31348ffdfd3 P bbe00c6771664c5393295b94a89bc4ae: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:44.325623 13144 tablet_bootstrap.cc:492] T 2fb3c3efe2374e68bff8a31348ffdfd3 P b1d47aae2b464377b14cd3ac5e8f1d2c: No bootstrap required, opened a new log
I20260925 22:05:44.325676 13144 ts_tablet_manager.cc:1397] T 2fb3c3efe2374e68bff8a31348ffdfd3 P b1d47aae2b464377b14cd3ac5e8f1d2c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 22:05:44.325758 13140 tablet_bootstrap.cc:492] T 2fb3c3efe2374e68bff8a31348ffdfd3 P 87968f19e8d8461aa549ca61f020e42f: No bootstrap required, opened a new log
I20260925 22:05:44.325801 13140 ts_tablet_manager.cc:1397] T 2fb3c3efe2374e68bff8a31348ffdfd3 P 87968f19e8d8461aa549ca61f020e42f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 22:05:44.325796 13144 raft_consensus.cc:348] T 2fb3c3efe2374e68bff8a31348ffdfd3 P b1d47aae2b464377b14cd3ac5e8f1d2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:44.325870 13144 raft_consensus.cc:374] T 2fb3c3efe2374e68bff8a31348ffdfd3 P b1d47aae2b464377b14cd3ac5e8f1d2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:44.325894 13144 raft_consensus.cc:729] T 2fb3c3efe2374e68bff8a31348ffdfd3 P b1d47aae2b464377b14cd3ac5e8f1d2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1d47aae2b464377b14cd3ac5e8f1d2c, State: Initialized, Role: FOLLOWER
I20260925 22:05:44.325918 13139 tablet_bootstrap.cc:492] T 2fb3c3efe2374e68bff8a31348ffdfd3 P bbe00c6771664c5393295b94a89bc4ae: No bootstrap required, opened a new log
I20260925 22:05:44.325958 13139 ts_tablet_manager.cc:1397] T 2fb3c3efe2374e68bff8a31348ffdfd3 P bbe00c6771664c5393295b94a89bc4ae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 22:05:44.325944 13144 consensus_queue.cc:260] T 2fb3c3efe2374e68bff8a31348ffdfd3 P b1d47aae2b464377b14cd3ac5e8f1d2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:44.325919 13140 raft_consensus.cc:348] T 2fb3c3efe2374e68bff8a31348ffdfd3 P 87968f19e8d8461aa549ca61f020e42f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:44.325999 13140 raft_consensus.cc:374] T 2fb3c3efe2374e68bff8a31348ffdfd3 P 87968f19e8d8461aa549ca61f020e42f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:44.326022 13140 raft_consensus.cc:729] T 2fb3c3efe2374e68bff8a31348ffdfd3 P 87968f19e8d8461aa549ca61f020e42f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87968f19e8d8461aa549ca61f020e42f, State: Initialized, Role: FOLLOWER
I20260925 22:05:44.326042 13144 ts_tablet_manager.cc:1428] T 2fb3c3efe2374e68bff8a31348ffdfd3 P b1d47aae2b464377b14cd3ac5e8f1d2c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:44.326064 13140 consensus_queue.cc:260] T 2fb3c3efe2374e68bff8a31348ffdfd3 P 87968f19e8d8461aa549ca61f020e42f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:44.326061 13157 tablet_replica.cc:439] T 2fb3c3efe2374e68bff8a31348ffdfd3 P b1d47aae2b464377b14cd3ac5e8f1d2c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } } }
I20260925 22:05:44.326094 13139 raft_consensus.cc:348] T 2fb3c3efe2374e68bff8a31348ffdfd3 P bbe00c6771664c5393295b94a89bc4ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:44.326154 13139 raft_consensus.cc:374] T 2fb3c3efe2374e68bff8a31348ffdfd3 P bbe00c6771664c5393295b94a89bc4ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:44.326140 13157 tablet_replica.cc:442] T 2fb3c3efe2374e68bff8a31348ffdfd3 P b1d47aae2b464377b14cd3ac5e8f1d2c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:05:44.326152 13140 ts_tablet_manager.cc:1428] T 2fb3c3efe2374e68bff8a31348ffdfd3 P 87968f19e8d8461aa549ca61f020e42f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:44.326175 13139 raft_consensus.cc:729] T 2fb3c3efe2374e68bff8a31348ffdfd3 P bbe00c6771664c5393295b94a89bc4ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbe00c6771664c5393295b94a89bc4ae, State: Initialized, Role: FOLLOWER
I20260925 22:05:44.326227 13139 consensus_queue.cc:260] T 2fb3c3efe2374e68bff8a31348ffdfd3 P bbe00c6771664c5393295b94a89bc4ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:44.326205 13145 tablet_replica.cc:439] T 2fb3c3efe2374e68bff8a31348ffdfd3 P 87968f19e8d8461aa549ca61f020e42f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } } }
I20260925 22:05:44.326280 13145 tablet_replica.cc:442] T 2fb3c3efe2374e68bff8a31348ffdfd3 P 87968f19e8d8461aa549ca61f020e42f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:05:44.326314 13139 ts_tablet_manager.cc:1428] T 2fb3c3efe2374e68bff8a31348ffdfd3 P bbe00c6771664c5393295b94a89bc4ae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:44.326470 13143 tablet_replica.cc:439] T 2fb3c3efe2374e68bff8a31348ffdfd3 P bbe00c6771664c5393295b94a89bc4ae: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } } }
I20260925 22:05:44.326553 13143 tablet_replica.cc:442] T 2fb3c3efe2374e68bff8a31348ffdfd3 P bbe00c6771664c5393295b94a89bc4ae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:05:44.364442 13145 raft_consensus.cc:482] T 2fb3c3efe2374e68bff8a31348ffdfd3 P 87968f19e8d8461aa549ca61f020e42f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:05:44.364547 13145 raft_consensus.cc:504] T 2fb3c3efe2374e68bff8a31348ffdfd3 P 87968f19e8d8461aa549ca61f020e42f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:44.364907 13145 leader_election.cc:288] T 2fb3c3efe2374e68bff8a31348ffdfd3 P 87968f19e8d8461aa549ca61f020e42f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bbe00c6771664c5393295b94a89bc4ae (127.5.0.66:40453), b1d47aae2b464377b14cd3ac5e8f1d2c (127.5.0.67:32885)
I20260925 22:05:44.368206 13137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:44.365151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:46533 (local address 127.5.0.67:32885)
0925 22:05:44.365195 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:44.365197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:44.365363 (+ 166us) server_negotiation.cc:407] Connection header received
0925 22:05:44.365397 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:44.365398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:44.365412 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:44.365470 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:44.365655 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.366406 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.367470 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.367822 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.367873 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:44.367881 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:44.368082 (+ 201us) server_negotiation.cc:299] Negotiation successful
0925 22:05:44.368127 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 22:05:44.368229 13183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:44.365065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:32885 (local address 127.5.0.65:46533)
0925 22:05:44.365286 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:44.365297 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:44.365351 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:44.365483 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:44.365485 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:44.365487 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:44.365581 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:44.365583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.366499 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.366500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:44.367376 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:44.367378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.367932 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.367933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:44.368002 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:44.368007 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:05:44.368060 (+ 53us) client_negotiation.cc:240] Negotiation successful
0925 22:05:44.368111 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":54,"thread_start_us":52,"threads_started":1}
I20260925 22:05:44.368258 13184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:44.365109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:44651 (local address 127.5.0.66:40453)
0925 22:05:44.365229 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:44.365231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:44.365300 (+ 69us) server_negotiation.cc:407] Connection header received
0925 22:05:44.365381 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:44.365382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:44.365394 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:44.365468 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:05:44.365655 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.366421 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.367471 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.367886 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.367980 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:44.367992 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:44.368128 (+ 136us) server_negotiation.cc:299] Negotiation successful
0925 22:05:44.368171 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":41,"threads_started":1}
I20260925 22:05:44.368237 13182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:44.365058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:40453 (local address 127.5.0.65:44651)
0925 22:05:44.365207 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:44.365218 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:44.365307 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:44.365486 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:44.365486 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:44.365487 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:05:44.365581 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:44.365583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.366559 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.366560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:44.367378 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:44.367380 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.367979 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.367980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:44.368035 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:44.368038 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:05:44.368112 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 22:05:44.368148 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":68,"thread_start_us":63,"threads_started":1}
I20260925 22:05:44.368484 13065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fb3c3efe2374e68bff8a31348ffdfd3" candidate_uuid: "87968f19e8d8461aa549ca61f020e42f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" is_pre_election: true
I20260925 22:05:44.368566 13065 raft_consensus.cc:2454] T 2fb3c3efe2374e68bff8a31348ffdfd3 P b1d47aae2b464377b14cd3ac5e8f1d2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87968f19e8d8461aa549ca61f020e42f in term 0.
I20260925 22:05:44.368676 12908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fb3c3efe2374e68bff8a31348ffdfd3" candidate_uuid: "87968f19e8d8461aa549ca61f020e42f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbe00c6771664c5393295b94a89bc4ae" is_pre_election: true
I20260925 22:05:44.368744 12717 leader_election.cc:302] T 2fb3c3efe2374e68bff8a31348ffdfd3 P 87968f19e8d8461aa549ca61f020e42f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87968f19e8d8461aa549ca61f020e42f, b1d47aae2b464377b14cd3ac5e8f1d2c; no voters:
I20260925 22:05:44.368784 12908 raft_consensus.cc:2454] T 2fb3c3efe2374e68bff8a31348ffdfd3 P bbe00c6771664c5393295b94a89bc4ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87968f19e8d8461aa549ca61f020e42f in term 0.
I20260925 22:05:44.368861 13145 raft_consensus.cc:2781] T 2fb3c3efe2374e68bff8a31348ffdfd3 P 87968f19e8d8461aa549ca61f020e42f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:05:44.368897 13145 raft_consensus.cc:482] T 2fb3c3efe2374e68bff8a31348ffdfd3 P 87968f19e8d8461aa549ca61f020e42f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:05:44.368923 13145 raft_consensus.cc:3037] T 2fb3c3efe2374e68bff8a31348ffdfd3 P 87968f19e8d8461aa549ca61f020e42f [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:44.369598 13145 raft_consensus.cc:504] T 2fb3c3efe2374e68bff8a31348ffdfd3 P 87968f19e8d8461aa549ca61f020e42f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:44.369747 13145 leader_election.cc:288] T 2fb3c3efe2374e68bff8a31348ffdfd3 P 87968f19e8d8461aa549ca61f020e42f [CANDIDATE]: Term 1 election: Requested vote from peers bbe00c6771664c5393295b94a89bc4ae (127.5.0.66:40453), b1d47aae2b464377b14cd3ac5e8f1d2c (127.5.0.67:32885)
I20260925 22:05:44.369875 13065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fb3c3efe2374e68bff8a31348ffdfd3" candidate_uuid: "87968f19e8d8461aa549ca61f020e42f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c"
I20260925 22:05:44.369875 12908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fb3c3efe2374e68bff8a31348ffdfd3" candidate_uuid: "87968f19e8d8461aa549ca61f020e42f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbe00c6771664c5393295b94a89bc4ae"
I20260925 22:05:44.369946 13065 raft_consensus.cc:3037] T 2fb3c3efe2374e68bff8a31348ffdfd3 P b1d47aae2b464377b14cd3ac5e8f1d2c [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:44.369946 12908 raft_consensus.cc:3037] T 2fb3c3efe2374e68bff8a31348ffdfd3 P bbe00c6771664c5393295b94a89bc4ae [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:44.370640 12908 raft_consensus.cc:2454] T 2fb3c3efe2374e68bff8a31348ffdfd3 P bbe00c6771664c5393295b94a89bc4ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87968f19e8d8461aa549ca61f020e42f in term 1.
I20260925 22:05:44.370639 13065 raft_consensus.cc:2454] T 2fb3c3efe2374e68bff8a31348ffdfd3 P b1d47aae2b464377b14cd3ac5e8f1d2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87968f19e8d8461aa549ca61f020e42f in term 1.
I20260925 22:05:44.370805 12718 leader_election.cc:302] T 2fb3c3efe2374e68bff8a31348ffdfd3 P 87968f19e8d8461aa549ca61f020e42f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87968f19e8d8461aa549ca61f020e42f, bbe00c6771664c5393295b94a89bc4ae; no voters:
I20260925 22:05:44.370904 13145 raft_consensus.cc:2781] T 2fb3c3efe2374e68bff8a31348ffdfd3 P 87968f19e8d8461aa549ca61f020e42f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:44.371001 13145 raft_consensus.cc:686] T 2fb3c3efe2374e68bff8a31348ffdfd3 P 87968f19e8d8461aa549ca61f020e42f [term 1 LEADER]: Becoming Leader. State: Replica: 87968f19e8d8461aa549ca61f020e42f, State: Running, Role: LEADER
I20260925 22:05:44.371071 13145 consensus_queue.cc:237] T 2fb3c3efe2374e68bff8a31348ffdfd3 P 87968f19e8d8461aa549ca61f020e42f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:44.371333 13185 tablet_replica.cc:439] T 2fb3c3efe2374e68bff8a31348ffdfd3 P 87968f19e8d8461aa549ca61f020e42f: TxnStatusTablet state changed. Reason: New leader 87968f19e8d8461aa549ca61f020e42f. Latest consensus state: current_term: 1 leader_uuid: "87968f19e8d8461aa549ca61f020e42f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } } }
I20260925 22:05:44.371423 13185 tablet_replica.cc:442] T 2fb3c3efe2374e68bff8a31348ffdfd3 P 87968f19e8d8461aa549ca61f020e42f: This TxnStatusTablet replica's current role is: LEADER
I20260925 22:05:44.371632 12628 catalog_manager.cc:5495] T 2fb3c3efe2374e68bff8a31348ffdfd3 P 87968f19e8d8461aa549ca61f020e42f reported cstate change: term changed from 0 to 1, leader changed from <none> to 87968f19e8d8461aa549ca61f020e42f (127.5.0.65). New cstate: current_term: 1 leader_uuid: "87968f19e8d8461aa549ca61f020e42f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } health_report { overall_health: UNKNOWN } } }
I20260925 22:05:44.371940 13187 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 22:05:44.372047 13187 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 22:05:44.387084 13138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:44.384476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60010 (local address 127.5.0.65:33097)
0925 22:05:44.384541 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:44.384543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:44.384553 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:05:44.384587 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:44.384588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:44.384605 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:44.384686 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:05:44.384769 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.385494 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.386183 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.386497 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.386574 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:44.386742 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:05:44.386840 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:05:44.386924 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:44.386994 (+ 70us) server_negotiation.cc:299] Negotiation successful
0925 22:05:44.387012 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 22:05:44.387096 13177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:44.384380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:33097 (local address 127.0.0.1:60010)
0925 22:05:44.384453 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:44.384467 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:44.384521 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:44.384685 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:44.384686 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:44.384687 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 22:05:44.384729 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:44.384731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.385561 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.385562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:44.386109 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:44.386110 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.386577 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.386578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:44.386615 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:44.386627 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 22:05:44.386923 (+ 296us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 22:05:44.386931 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 22:05:44.386980 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 22:05:44.387031 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 22:05:44.388269 12908 raft_consensus.cc:1261] T 2fb3c3efe2374e68bff8a31348ffdfd3 P bbe00c6771664c5393295b94a89bc4ae [term 1 FOLLOWER]: Refusing update from remote peer 87968f19e8d8461aa549ca61f020e42f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:05:44.388302 13065 raft_consensus.cc:1261] T 2fb3c3efe2374e68bff8a31348ffdfd3 P b1d47aae2b464377b14cd3ac5e8f1d2c [term 1 FOLLOWER]: Refusing update from remote peer 87968f19e8d8461aa549ca61f020e42f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:05:44.388504 13185 consensus_queue.cc:1035] T 2fb3c3efe2374e68bff8a31348ffdfd3 P 87968f19e8d8461aa549ca61f020e42f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:05:44.388600 13145 consensus_queue.cc:1035] T 2fb3c3efe2374e68bff8a31348ffdfd3 P 87968f19e8d8461aa549ca61f020e42f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:05:44.389132 13143 tablet_replica.cc:439] T 2fb3c3efe2374e68bff8a31348ffdfd3 P bbe00c6771664c5393295b94a89bc4ae: TxnStatusTablet state changed. Reason: New leader 87968f19e8d8461aa549ca61f020e42f. Latest consensus state: current_term: 1 leader_uuid: "87968f19e8d8461aa549ca61f020e42f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } } }
I20260925 22:05:44.389221 13143 tablet_replica.cc:442] T 2fb3c3efe2374e68bff8a31348ffdfd3 P bbe00c6771664c5393295b94a89bc4ae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:05:44.389271 13157 tablet_replica.cc:439] T 2fb3c3efe2374e68bff8a31348ffdfd3 P b1d47aae2b464377b14cd3ac5e8f1d2c: TxnStatusTablet state changed. Reason: New leader 87968f19e8d8461aa549ca61f020e42f. Latest consensus state: current_term: 1 leader_uuid: "87968f19e8d8461aa549ca61f020e42f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } } }
I20260925 22:05:44.389580 13157 tablet_replica.cc:442] T 2fb3c3efe2374e68bff8a31348ffdfd3 P b1d47aae2b464377b14cd3ac5e8f1d2c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:05:44.390431 13145 tablet_replica.cc:439] T 2fb3c3efe2374e68bff8a31348ffdfd3 P 87968f19e8d8461aa549ca61f020e42f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "87968f19e8d8461aa549ca61f020e42f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } } }
I20260925 22:05:44.390517 13145 tablet_replica.cc:442] T 2fb3c3efe2374e68bff8a31348ffdfd3 P 87968f19e8d8461aa549ca61f020e42f: This TxnStatusTablet replica's current role is: LEADER
I20260925 22:05:44.390786 13157 tablet_replica.cc:439] T 2fb3c3efe2374e68bff8a31348ffdfd3 P b1d47aae2b464377b14cd3ac5e8f1d2c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "87968f19e8d8461aa549ca61f020e42f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } } }
I20260925 22:05:44.390864 13157 tablet_replica.cc:442] T 2fb3c3efe2374e68bff8a31348ffdfd3 P b1d47aae2b464377b14cd3ac5e8f1d2c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:05:44.394448 13184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:44.391991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36778 (local address 127.5.0.66:40453)
0925 22:05:44.392031 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:44.392033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:44.392136 (+ 103us) server_negotiation.cc:407] Connection header received
0925 22:05:44.392224 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:44.392224 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:44.392245 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:44.392331 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:05:44.392452 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.392901 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.393670 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.393875 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.393950 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:44.394121 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:05:44.394225 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:05:44.394308 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:44.394349 (+ 41us) server_negotiation.cc:299] Negotiation successful
0925 22:05:44.394370 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:05:44.396445 13145 tablet_replica.cc:439] T 2fb3c3efe2374e68bff8a31348ffdfd3 P 87968f19e8d8461aa549ca61f020e42f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "87968f19e8d8461aa549ca61f020e42f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } } }
I20260925 22:05:44.396533 13145 tablet_replica.cc:442] T 2fb3c3efe2374e68bff8a31348ffdfd3 P 87968f19e8d8461aa549ca61f020e42f: This TxnStatusTablet replica's current role is: LEADER
I20260925 22:05:44.397048 13143 tablet_replica.cc:439] T 2fb3c3efe2374e68bff8a31348ffdfd3 P bbe00c6771664c5393295b94a89bc4ae: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "87968f19e8d8461aa549ca61f020e42f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } } }
I20260925 22:05:44.397131 13143 tablet_replica.cc:442] T 2fb3c3efe2374e68bff8a31348ffdfd3 P bbe00c6771664c5393295b94a89bc4ae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:05:44.403414 13138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:44.400831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:51577 (local address 127.5.0.65:33097)
0925 22:05:44.400898 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:44.400900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:44.401039 (+ 139us) server_negotiation.cc:407] Connection header received
0925 22:05:44.401106 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:44.401107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:44.401119 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:44.401167 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:05:44.401290 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.401850 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.402680 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.402894 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.402974 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:44.403144 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:05:44.403178 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:05:44.403252 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:44.403315 (+ 63us) server_negotiation.cc:299] Negotiation successful
0925 22:05:44.403336 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260925 22:05:44.403432 13203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:44.400761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:33097 (local address 127.5.0.66:51577)
0925 22:05:44.400969 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:44.400979 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:44.401054 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:44.401167 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:44.401167 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:44.401169 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 22:05:44.401229 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:44.401230 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.401954 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.401956 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:44.402611 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:44.402614 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.402974 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.402974 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:44.403039 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:44.403063 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 22:05:44.403254 (+ 191us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 22:05:44.403257 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:05:44.403312 (+ 55us) client_negotiation.cc:240] Negotiation successful
0925 22:05:44.403354 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":94,"threads_started":1}
I20260925 22:05:44.405740 13065 raft_consensus.cc:1261] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P b1d47aae2b464377b14cd3ac5e8f1d2c [term 1 FOLLOWER]: Refusing update from remote peer bbe00c6771664c5393295b94a89bc4ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:05:44.405771 12785 raft_consensus.cc:1261] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P 87968f19e8d8461aa549ca61f020e42f [term 1 FOLLOWER]: Refusing update from remote peer bbe00c6771664c5393295b94a89bc4ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:05:44.405988 13143 consensus_queue.cc:1035] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P bbe00c6771664c5393295b94a89bc4ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:05:44.406118 13170 consensus_queue.cc:1035] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P bbe00c6771664c5393295b94a89bc4ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:05:44.408092 13190 mvcc.cc:205] Tried to move back new op lower bound from 7333371676284162048 to 7333371675528916992. Current Snapshot: MvccSnapshot[applied={T|T < 7333371676284162048}]
I20260925 22:05:44.414687 13184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:44.412007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:52047 (local address 127.5.0.66:40453)
0925 22:05:44.412062 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:44.412064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:44.412106 (+ 42us) server_negotiation.cc:407] Connection header received
0925 22:05:44.412177 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:44.412178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:44.412185 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:44.412232 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:05:44.412362 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.412821 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.413947 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.414124 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.414172 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:44.414373 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:05:44.414440 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:05:44.414495 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:44.414563 (+ 68us) server_negotiation.cc:299] Negotiation successful
0925 22:05:44.414613 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 22:05:44.414743 13207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:44.411826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:40453 (local address 127.5.0.65:52047)
0925 22:05:44.412051 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:44.412062 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:44.412119 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:44.412236 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:44.412237 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:44.412238 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 22:05:44.412307 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:44.412309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.412891 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.412892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:44.413861 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:44.413864 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.414212 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.414213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:44.414276 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:44.414291 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 22:05:44.414505 (+ 214us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 22:05:44.414506 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 22:05:44.414560 (+ 54us) client_negotiation.cc:240] Negotiation successful
0925 22:05:44.414603 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":89,"threads_started":1}
I20260925 22:05:44.423709 12627 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:05:44.423820 12627 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:05:44.424453 12627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f5a748e94d32429ca130f24868a54d8c: Sending DeleteTablet for 3 replicas of tablet 3dd9b6f5b6ac40a1a1a41efa43c2674b
I20260925 22:05:44.424738 13045 tablet_service.cc:1514] Processing DeleteTablet for tablet 3dd9b6f5b6ac40a1a1a41efa43c2674b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:44 UTC) from {username='oryx', principal='oryx/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:54710
I20260925 22:05:44.424775 12765 tablet_service.cc:1514] Processing DeleteTablet for tablet 3dd9b6f5b6ac40a1a1a41efa43c2674b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:44 UTC) from {username='oryx', principal='oryx/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:60000
I20260925 22:05:44.424964 13211 tablet_replica.cc:330] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P b1d47aae2b464377b14cd3ac5e8f1d2c: stopping tablet replica
I20260925 22:05:44.425028 13211 raft_consensus.cc:2229] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P b1d47aae2b464377b14cd3ac5e8f1d2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:05:44.425063 13211 raft_consensus.cc:2258] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P b1d47aae2b464377b14cd3ac5e8f1d2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:44.425319 13211 ts_tablet_manager.cc:1905] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P b1d47aae2b464377b14cd3ac5e8f1d2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:44.427048 13212 tablet_replica.cc:330] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P 87968f19e8d8461aa549ca61f020e42f: stopping tablet replica
I20260925 22:05:44.427114 13212 raft_consensus.cc:2229] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P 87968f19e8d8461aa549ca61f020e42f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:05:44.427150 13212 raft_consensus.cc:2258] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P 87968f19e8d8461aa549ca61f020e42f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:44.427269 13211 ts_tablet_manager.cc:1918] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P b1d47aae2b464377b14cd3ac5e8f1d2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 22:05:44.427313 13211 log.cc:1196] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P b1d47aae2b464377b14cd3ac5e8f1d2c: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-2/wal/wals/3dd9b6f5b6ac40a1a1a41efa43c2674b
I20260925 22:05:44.427579 13211 ts_tablet_manager.cc:1939] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P b1d47aae2b464377b14cd3ac5e8f1d2c: Deleting consensus metadata
I20260925 22:05:44.428030 13212 ts_tablet_manager.cc:1905] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P 87968f19e8d8461aa549ca61f020e42f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:44.428299 12613 catalog_manager.cc:4841] TS b1d47aae2b464377b14cd3ac5e8f1d2c (127.5.0.67:32885): tablet 3dd9b6f5b6ac40a1a1a41efa43c2674b (table test-table [id=c6281e95f2e7463591552b0eb395ed6b]) successfully deleted
I20260925 22:05:44.428687 12905 tablet_service.cc:1514] Processing DeleteTablet for tablet 3dd9b6f5b6ac40a1a1a41efa43c2674b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:44 UTC) from {username='oryx', principal='oryx/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:36776
I20260925 22:05:44.429543 13212 ts_tablet_manager.cc:1918] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P 87968f19e8d8461aa549ca61f020e42f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 22:05:44.429605 13212 log.cc:1196] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P 87968f19e8d8461aa549ca61f020e42f: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-0/wal/wals/3dd9b6f5b6ac40a1a1a41efa43c2674b
I20260925 22:05:44.430164 13212 ts_tablet_manager.cc:1939] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P 87968f19e8d8461aa549ca61f020e42f: Deleting consensus metadata
I20260925 22:05:44.430361 13214 tablet_replica.cc:330] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P bbe00c6771664c5393295b94a89bc4ae: stopping tablet replica
I20260925 22:05:44.430456 13214 raft_consensus.cc:2229] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P bbe00c6771664c5393295b94a89bc4ae [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:05:44.430466 12613 catalog_manager.cc:4841] TS 87968f19e8d8461aa549ca61f020e42f (127.5.0.65:33097): tablet 3dd9b6f5b6ac40a1a1a41efa43c2674b (table test-table [id=c6281e95f2e7463591552b0eb395ed6b]) successfully deleted
I20260925 22:05:44.430624 13214 raft_consensus.cc:2258] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P bbe00c6771664c5393295b94a89bc4ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:44.430941 13214 ts_tablet_manager.cc:1905] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P bbe00c6771664c5393295b94a89bc4ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:44.432659 13214 ts_tablet_manager.cc:1918] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P bbe00c6771664c5393295b94a89bc4ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 22:05:44.432788 13214 log.cc:1196] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P bbe00c6771664c5393295b94a89bc4ae: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-1/wal/wals/3dd9b6f5b6ac40a1a1a41efa43c2674b
I20260925 22:05:44.433064 13214 ts_tablet_manager.cc:1939] T 3dd9b6f5b6ac40a1a1a41efa43c2674b P bbe00c6771664c5393295b94a89bc4ae: Deleting consensus metadata
I20260925 22:05:44.433413 12614 catalog_manager.cc:4841] TS bbe00c6771664c5393295b94a89bc4ae (127.5.0.66:40453): tablet 3dd9b6f5b6ac40a1a1a41efa43c2674b (table test-table [id=c6281e95f2e7463591552b0eb395ed6b]) successfully deleted
Sep 25 22:05:44 dist-test-slave-7mqr krb5kdc[12583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 22:05:44 dist-test-slave-7mqr krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:44 dist-test-slave-7mqr krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
W20260925 22:05:44.457576 13178 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:44.450420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35112 (local address 127.5.0.126:45363)
0925 22:05:44.450475 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:44.450477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:44.450667 (+ 190us) server_negotiation.cc:407] Connection header received
0925 22:05:44.450712 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:44.450714 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:44.450728 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:44.450800 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:44.451589 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.452497 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.453573 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.453908 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.453983 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:44.457506 (+ 3523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35112: BlockingRecv error: recv got EOF from 127.0.0.1:35112 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Sep 25 22:05:44 dist-test-slave-7mqr krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM
I20260925 22:05:44.467530 13178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:44.458725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35126 (local address 127.5.0.126:45363)
0925 22:05:44.458763 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:44.458765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:44.459001 (+ 236us) server_negotiation.cc:407] Connection header received
0925 22:05:44.459051 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:44.459052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:44.459063 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:44.459125 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:44.459710 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.460459 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.461480 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.461777 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.461853 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:44.464540 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:44.464541 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:44.464542 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:44.464549 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:44.466267 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:44.466680 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:44.466681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:44.466681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:44.466751 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:44.466968 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:44.466968 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:44.466969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:44.467102 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:44.467201 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:44.467350 (+ 149us) server_negotiation.cc:299] Negotiation successful
0925 22:05:44.467431 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:05:44.469496 12627 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35126:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:05:44.469663 12627 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:05:44.471105 12765 tablet_service.cc:1467] Processing CreateTablet for tablet acb132d2a8fb489aa8f207355272ff1a (DEFAULT_TABLE table=test-table [id=63e8904beefe4e3e997e8beff0ed85bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:44.471238 12765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acb132d2a8fb489aa8f207355272ff1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:44.471978 12905 tablet_service.cc:1467] Processing CreateTablet for tablet acb132d2a8fb489aa8f207355272ff1a (DEFAULT_TABLE table=test-table [id=63e8904beefe4e3e997e8beff0ed85bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:44.472116 12905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acb132d2a8fb489aa8f207355272ff1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:44.473390 13045 tablet_service.cc:1467] Processing CreateTablet for tablet acb132d2a8fb489aa8f207355272ff1a (DEFAULT_TABLE table=test-table [id=63e8904beefe4e3e997e8beff0ed85bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:44.473500 13045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acb132d2a8fb489aa8f207355272ff1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:44.474749 13139 tablet_bootstrap.cc:492] T acb132d2a8fb489aa8f207355272ff1a P bbe00c6771664c5393295b94a89bc4ae: Bootstrap starting.
I20260925 22:05:44.475154 13139 tablet_bootstrap.cc:654] T acb132d2a8fb489aa8f207355272ff1a P bbe00c6771664c5393295b94a89bc4ae: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:44.475477 13144 tablet_bootstrap.cc:492] T acb132d2a8fb489aa8f207355272ff1a P b1d47aae2b464377b14cd3ac5e8f1d2c: Bootstrap starting.
I20260925 22:05:44.475777 13144 tablet_bootstrap.cc:654] T acb132d2a8fb489aa8f207355272ff1a P b1d47aae2b464377b14cd3ac5e8f1d2c: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:44.476653 13144 tablet_bootstrap.cc:492] T acb132d2a8fb489aa8f207355272ff1a P b1d47aae2b464377b14cd3ac5e8f1d2c: No bootstrap required, opened a new log
I20260925 22:05:44.476709 13144 ts_tablet_manager.cc:1397] T acb132d2a8fb489aa8f207355272ff1a P b1d47aae2b464377b14cd3ac5e8f1d2c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 22:05:44.476846 13144 raft_consensus.cc:348] T acb132d2a8fb489aa8f207355272ff1a P b1d47aae2b464377b14cd3ac5e8f1d2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:44.476909 13144 raft_consensus.cc:374] T acb132d2a8fb489aa8f207355272ff1a P b1d47aae2b464377b14cd3ac5e8f1d2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:44.476931 13144 raft_consensus.cc:729] T acb132d2a8fb489aa8f207355272ff1a P b1d47aae2b464377b14cd3ac5e8f1d2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1d47aae2b464377b14cd3ac5e8f1d2c, State: Initialized, Role: FOLLOWER
I20260925 22:05:44.476971 13144 consensus_queue.cc:260] T acb132d2a8fb489aa8f207355272ff1a P b1d47aae2b464377b14cd3ac5e8f1d2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:44.477051 13144 ts_tablet_manager.cc:1428] T acb132d2a8fb489aa8f207355272ff1a P b1d47aae2b464377b14cd3ac5e8f1d2c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:44.477662 13139 tablet_bootstrap.cc:492] T acb132d2a8fb489aa8f207355272ff1a P bbe00c6771664c5393295b94a89bc4ae: No bootstrap required, opened a new log
I20260925 22:05:44.477722 13139 ts_tablet_manager.cc:1397] T acb132d2a8fb489aa8f207355272ff1a P bbe00c6771664c5393295b94a89bc4ae: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260925 22:05:44.477856 13139 raft_consensus.cc:348] T acb132d2a8fb489aa8f207355272ff1a P bbe00c6771664c5393295b94a89bc4ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:44.477916 13139 raft_consensus.cc:374] T acb132d2a8fb489aa8f207355272ff1a P bbe00c6771664c5393295b94a89bc4ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:44.477936 13139 raft_consensus.cc:729] T acb132d2a8fb489aa8f207355272ff1a P bbe00c6771664c5393295b94a89bc4ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbe00c6771664c5393295b94a89bc4ae, State: Initialized, Role: FOLLOWER
I20260925 22:05:44.477979 13139 consensus_queue.cc:260] T acb132d2a8fb489aa8f207355272ff1a P bbe00c6771664c5393295b94a89bc4ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:44.478053 13139 ts_tablet_manager.cc:1428] T acb132d2a8fb489aa8f207355272ff1a P bbe00c6771664c5393295b94a89bc4ae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:44.480084 13140 tablet_bootstrap.cc:492] T acb132d2a8fb489aa8f207355272ff1a P 87968f19e8d8461aa549ca61f020e42f: Bootstrap starting.
I20260925 22:05:44.480466 13140 tablet_bootstrap.cc:654] T acb132d2a8fb489aa8f207355272ff1a P 87968f19e8d8461aa549ca61f020e42f: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:44.481225 13140 tablet_bootstrap.cc:492] T acb132d2a8fb489aa8f207355272ff1a P 87968f19e8d8461aa549ca61f020e42f: No bootstrap required, opened a new log
I20260925 22:05:44.481273 13140 ts_tablet_manager.cc:1397] T acb132d2a8fb489aa8f207355272ff1a P 87968f19e8d8461aa549ca61f020e42f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 22:05:44.481555 13140 raft_consensus.cc:348] T acb132d2a8fb489aa8f207355272ff1a P 87968f19e8d8461aa549ca61f020e42f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:44.481617 13140 raft_consensus.cc:374] T acb132d2a8fb489aa8f207355272ff1a P 87968f19e8d8461aa549ca61f020e42f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:44.481637 13140 raft_consensus.cc:729] T acb132d2a8fb489aa8f207355272ff1a P 87968f19e8d8461aa549ca61f020e42f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87968f19e8d8461aa549ca61f020e42f, State: Initialized, Role: FOLLOWER
I20260925 22:05:44.481674 13140 consensus_queue.cc:260] T acb132d2a8fb489aa8f207355272ff1a P 87968f19e8d8461aa549ca61f020e42f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:44.481784 13140 ts_tablet_manager.cc:1428] T acb132d2a8fb489aa8f207355272ff1a P 87968f19e8d8461aa549ca61f020e42f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:44.692075 13157 raft_consensus.cc:482] T acb132d2a8fb489aa8f207355272ff1a P b1d47aae2b464377b14cd3ac5e8f1d2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:05:44.692193 13157 raft_consensus.cc:504] T acb132d2a8fb489aa8f207355272ff1a P b1d47aae2b464377b14cd3ac5e8f1d2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:44.692408 13157 leader_election.cc:288] T acb132d2a8fb489aa8f207355272ff1a P b1d47aae2b464377b14cd3ac5e8f1d2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87968f19e8d8461aa549ca61f020e42f (127.5.0.65:33097), bbe00c6771664c5393295b94a89bc4ae (127.5.0.66:40453)
I20260925 22:05:44.692668 12908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acb132d2a8fb489aa8f207355272ff1a" candidate_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbe00c6771664c5393295b94a89bc4ae" is_pre_election: true
I20260925 22:05:44.692667 12785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acb132d2a8fb489aa8f207355272ff1a" candidate_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87968f19e8d8461aa549ca61f020e42f" is_pre_election: true
I20260925 22:05:44.692781 12908 raft_consensus.cc:2454] T acb132d2a8fb489aa8f207355272ff1a P bbe00c6771664c5393295b94a89bc4ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1d47aae2b464377b14cd3ac5e8f1d2c in term 0.
I20260925 22:05:44.692780 12785 raft_consensus.cc:2454] T acb132d2a8fb489aa8f207355272ff1a P 87968f19e8d8461aa549ca61f020e42f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1d47aae2b464377b14cd3ac5e8f1d2c in term 0.
I20260925 22:05:44.692951 12997 leader_election.cc:302] T acb132d2a8fb489aa8f207355272ff1a P b1d47aae2b464377b14cd3ac5e8f1d2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87968f19e8d8461aa549ca61f020e42f, b1d47aae2b464377b14cd3ac5e8f1d2c; no voters:
I20260925 22:05:44.693068 13157 raft_consensus.cc:2781] T acb132d2a8fb489aa8f207355272ff1a P b1d47aae2b464377b14cd3ac5e8f1d2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:05:44.693121 13157 raft_consensus.cc:482] T acb132d2a8fb489aa8f207355272ff1a P b1d47aae2b464377b14cd3ac5e8f1d2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:05:44.693137 13157 raft_consensus.cc:3037] T acb132d2a8fb489aa8f207355272ff1a P b1d47aae2b464377b14cd3ac5e8f1d2c [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:44.693878 13157 raft_consensus.cc:504] T acb132d2a8fb489aa8f207355272ff1a P b1d47aae2b464377b14cd3ac5e8f1d2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:44.694012 13157 leader_election.cc:288] T acb132d2a8fb489aa8f207355272ff1a P b1d47aae2b464377b14cd3ac5e8f1d2c [CANDIDATE]: Term 1 election: Requested vote from peers 87968f19e8d8461aa549ca61f020e42f (127.5.0.65:33097), bbe00c6771664c5393295b94a89bc4ae (127.5.0.66:40453)
I20260925 22:05:44.694167 12785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acb132d2a8fb489aa8f207355272ff1a" candidate_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87968f19e8d8461aa549ca61f020e42f"
I20260925 22:05:44.694172 12908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acb132d2a8fb489aa8f207355272ff1a" candidate_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbe00c6771664c5393295b94a89bc4ae"
I20260925 22:05:44.694248 12908 raft_consensus.cc:3037] T acb132d2a8fb489aa8f207355272ff1a P bbe00c6771664c5393295b94a89bc4ae [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:44.694271 12785 raft_consensus.cc:3037] T acb132d2a8fb489aa8f207355272ff1a P 87968f19e8d8461aa549ca61f020e42f [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:44.694917 12785 raft_consensus.cc:2454] T acb132d2a8fb489aa8f207355272ff1a P 87968f19e8d8461aa549ca61f020e42f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1d47aae2b464377b14cd3ac5e8f1d2c in term 1.
I20260925 22:05:44.694986 12908 raft_consensus.cc:2454] T acb132d2a8fb489aa8f207355272ff1a P bbe00c6771664c5393295b94a89bc4ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1d47aae2b464377b14cd3ac5e8f1d2c in term 1.
I20260925 22:05:44.695181 12998 leader_election.cc:302] T acb132d2a8fb489aa8f207355272ff1a P b1d47aae2b464377b14cd3ac5e8f1d2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1d47aae2b464377b14cd3ac5e8f1d2c, bbe00c6771664c5393295b94a89bc4ae; no voters:
I20260925 22:05:44.695304 13157 raft_consensus.cc:2781] T acb132d2a8fb489aa8f207355272ff1a P b1d47aae2b464377b14cd3ac5e8f1d2c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:44.695461 13157 raft_consensus.cc:686] T acb132d2a8fb489aa8f207355272ff1a P b1d47aae2b464377b14cd3ac5e8f1d2c [term 1 LEADER]: Becoming Leader. State: Replica: b1d47aae2b464377b14cd3ac5e8f1d2c, State: Running, Role: LEADER
I20260925 22:05:44.695521 13157 consensus_queue.cc:237] T acb132d2a8fb489aa8f207355272ff1a P b1d47aae2b464377b14cd3ac5e8f1d2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:44.696120 12628 catalog_manager.cc:5495] T acb132d2a8fb489aa8f207355272ff1a P b1d47aae2b464377b14cd3ac5e8f1d2c reported cstate change: term changed from 0 to 1, leader changed from <none> to b1d47aae2b464377b14cd3ac5e8f1d2c (127.5.0.67). New cstate: current_term: 1 leader_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } health_report { overall_health: HEALTHY } } }
I20260925 22:05:44.710891 13137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:44.708706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54728 (local address 127.5.0.67:32885)
0925 22:05:44.708755 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:44.708756 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:44.708764 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:05:44.708814 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:44.708815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:44.708824 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:44.708872 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:05:44.708986 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.709453 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.710251 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.710424 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.710476 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:44.710655 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:05:44.710682 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:05:44.710719 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:44.710818 (+ 99us) server_negotiation.cc:299] Negotiation successful
0925 22:05:44.710834 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 22:05:44.711741 12785 raft_consensus.cc:1261] T acb132d2a8fb489aa8f207355272ff1a P 87968f19e8d8461aa549ca61f020e42f [term 1 FOLLOWER]: Refusing update from remote peer b1d47aae2b464377b14cd3ac5e8f1d2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:05:44.711748 12908 raft_consensus.cc:1261] T acb132d2a8fb489aa8f207355272ff1a P bbe00c6771664c5393295b94a89bc4ae [term 1 FOLLOWER]: Refusing update from remote peer b1d47aae2b464377b14cd3ac5e8f1d2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:05:44.712001 13229 consensus_queue.cc:1035] T acb132d2a8fb489aa8f207355272ff1a P b1d47aae2b464377b14cd3ac5e8f1d2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:05:44.712103 13157 consensus_queue.cc:1035] T acb132d2a8fb489aa8f207355272ff1a P b1d47aae2b464377b14cd3ac5e8f1d2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:05:44.714694 12628 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:05:44.714776 12628 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:05:44.715418 12628 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f5a748e94d32429ca130f24868a54d8c: Sending DeleteTablet for 3 replicas of tablet acb132d2a8fb489aa8f207355272ff1a
I20260925 22:05:44.715672 12765 tablet_service.cc:1514] Processing DeleteTablet for tablet acb132d2a8fb489aa8f207355272ff1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:44 UTC) from {username='oryx', principal='oryx/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:60000
I20260925 22:05:44.715665 13045 tablet_service.cc:1514] Processing DeleteTablet for tablet acb132d2a8fb489aa8f207355272ff1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:44 UTC) from {username='oryx', principal='oryx/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:54710
I20260925 22:05:44.715801 13211 tablet_replica.cc:330] T acb132d2a8fb489aa8f207355272ff1a P b1d47aae2b464377b14cd3ac5e8f1d2c: stopping tablet replica
I20260925 22:05:44.715821 13212 tablet_replica.cc:330] T acb132d2a8fb489aa8f207355272ff1a P 87968f19e8d8461aa549ca61f020e42f: stopping tablet replica
I20260925 22:05:44.715868 13212 raft_consensus.cc:2229] T acb132d2a8fb489aa8f207355272ff1a P 87968f19e8d8461aa549ca61f020e42f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:05:44.715867 13211 raft_consensus.cc:2229] T acb132d2a8fb489aa8f207355272ff1a P b1d47aae2b464377b14cd3ac5e8f1d2c [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:05:44.715847 12905 tablet_service.cc:1514] Processing DeleteTablet for tablet acb132d2a8fb489aa8f207355272ff1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:44 UTC) from {username='oryx', principal='oryx/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:36776
I20260925 22:05:44.715938 13212 raft_consensus.cc:2258] T acb132d2a8fb489aa8f207355272ff1a P 87968f19e8d8461aa549ca61f020e42f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:44.715952 13211 raft_consensus.cc:2258] T acb132d2a8fb489aa8f207355272ff1a P b1d47aae2b464377b14cd3ac5e8f1d2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:44.715960 13214 tablet_replica.cc:330] T acb132d2a8fb489aa8f207355272ff1a P bbe00c6771664c5393295b94a89bc4ae: stopping tablet replica
I20260925 22:05:44.716003 13214 raft_consensus.cc:2229] T acb132d2a8fb489aa8f207355272ff1a P bbe00c6771664c5393295b94a89bc4ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:05:44.716064 13214 raft_consensus.cc:2258] T acb132d2a8fb489aa8f207355272ff1a P bbe00c6771664c5393295b94a89bc4ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:44.716331 13214 ts_tablet_manager.cc:1905] T acb132d2a8fb489aa8f207355272ff1a P bbe00c6771664c5393295b94a89bc4ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:44.716451 13211 ts_tablet_manager.cc:1905] T acb132d2a8fb489aa8f207355272ff1a P b1d47aae2b464377b14cd3ac5e8f1d2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:44.716388 12628 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35126:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:05:44.716529 12628 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:05:44.717643 12765 tablet_service.cc:1467] Processing CreateTablet for tablet e05cd5545ef94b4eb2f6138eb01b8b9e (DEFAULT_TABLE table=test-table [id=62b917d508ac4caaac996c3b30abac7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:44.717682 13045 tablet_service.cc:1467] Processing CreateTablet for tablet e05cd5545ef94b4eb2f6138eb01b8b9e (DEFAULT_TABLE table=test-table [id=62b917d508ac4caaac996c3b30abac7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:44.717774 13045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e05cd5545ef94b4eb2f6138eb01b8b9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:44.717767 12765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e05cd5545ef94b4eb2f6138eb01b8b9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:44.717916 13214 ts_tablet_manager.cc:1918] T acb132d2a8fb489aa8f207355272ff1a P bbe00c6771664c5393295b94a89bc4ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:05:44.717962 13214 log.cc:1196] T acb132d2a8fb489aa8f207355272ff1a P bbe00c6771664c5393295b94a89bc4ae: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-1/wal/wals/acb132d2a8fb489aa8f207355272ff1a
I20260925 22:05:44.718124 13211 ts_tablet_manager.cc:1918] T acb132d2a8fb489aa8f207355272ff1a P b1d47aae2b464377b14cd3ac5e8f1d2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:05:44.718171 13211 log.cc:1196] T acb132d2a8fb489aa8f207355272ff1a P b1d47aae2b464377b14cd3ac5e8f1d2c: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-2/wal/wals/acb132d2a8fb489aa8f207355272ff1a
I20260925 22:05:44.718182 13214 ts_tablet_manager.cc:1939] T acb132d2a8fb489aa8f207355272ff1a P bbe00c6771664c5393295b94a89bc4ae: Deleting consensus metadata
I20260925 22:05:44.718394 13211 ts_tablet_manager.cc:1939] T acb132d2a8fb489aa8f207355272ff1a P b1d47aae2b464377b14cd3ac5e8f1d2c: Deleting consensus metadata
I20260925 22:05:44.718611 12614 catalog_manager.cc:4841] TS bbe00c6771664c5393295b94a89bc4ae (127.5.0.66:40453): tablet acb132d2a8fb489aa8f207355272ff1a (table test-table [id=63e8904beefe4e3e997e8beff0ed85bf]) successfully deleted
I20260925 22:05:44.718829 12905 tablet_service.cc:1467] Processing CreateTablet for tablet e05cd5545ef94b4eb2f6138eb01b8b9e (DEFAULT_TABLE table=test-table [id=62b917d508ac4caaac996c3b30abac7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:05:44.718878 13212 ts_tablet_manager.cc:1905] T acb132d2a8fb489aa8f207355272ff1a P 87968f19e8d8461aa549ca61f020e42f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:44.718923 12905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e05cd5545ef94b4eb2f6138eb01b8b9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:44.719988 13139 tablet_bootstrap.cc:492] T e05cd5545ef94b4eb2f6138eb01b8b9e P bbe00c6771664c5393295b94a89bc4ae: Bootstrap starting.
I20260925 22:05:44.720014 13144 tablet_bootstrap.cc:492] T e05cd5545ef94b4eb2f6138eb01b8b9e P b1d47aae2b464377b14cd3ac5e8f1d2c: Bootstrap starting.
I20260925 22:05:44.720120 13212 ts_tablet_manager.cc:1918] T acb132d2a8fb489aa8f207355272ff1a P 87968f19e8d8461aa549ca61f020e42f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:05:44.720178 13212 log.cc:1196] T acb132d2a8fb489aa8f207355272ff1a P 87968f19e8d8461aa549ca61f020e42f: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-0/wal/wals/acb132d2a8fb489aa8f207355272ff1a
I20260925 22:05:44.720288 13139 tablet_bootstrap.cc:654] T e05cd5545ef94b4eb2f6138eb01b8b9e P bbe00c6771664c5393295b94a89bc4ae: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:44.720351 13144 tablet_bootstrap.cc:654] T e05cd5545ef94b4eb2f6138eb01b8b9e P b1d47aae2b464377b14cd3ac5e8f1d2c: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:44.720412 13212 ts_tablet_manager.cc:1939] T acb132d2a8fb489aa8f207355272ff1a P 87968f19e8d8461aa549ca61f020e42f: Deleting consensus metadata
I20260925 22:05:44.720615 13140 tablet_bootstrap.cc:492] T e05cd5545ef94b4eb2f6138eb01b8b9e P 87968f19e8d8461aa549ca61f020e42f: Bootstrap starting.
I20260925 22:05:44.720892 13139 tablet_bootstrap.cc:492] T e05cd5545ef94b4eb2f6138eb01b8b9e P bbe00c6771664c5393295b94a89bc4ae: No bootstrap required, opened a new log
I20260925 22:05:44.720906 13140 tablet_bootstrap.cc:654] T e05cd5545ef94b4eb2f6138eb01b8b9e P 87968f19e8d8461aa549ca61f020e42f: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:44.720942 13139 ts_tablet_manager.cc:1397] T e05cd5545ef94b4eb2f6138eb01b8b9e P bbe00c6771664c5393295b94a89bc4ae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 22:05:44.721084 13139 raft_consensus.cc:348] T e05cd5545ef94b4eb2f6138eb01b8b9e P bbe00c6771664c5393295b94a89bc4ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:44.721176 13139 raft_consensus.cc:374] T e05cd5545ef94b4eb2f6138eb01b8b9e P bbe00c6771664c5393295b94a89bc4ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:44.721211 13139 raft_consensus.cc:729] T e05cd5545ef94b4eb2f6138eb01b8b9e P bbe00c6771664c5393295b94a89bc4ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbe00c6771664c5393295b94a89bc4ae, State: Initialized, Role: FOLLOWER
I20260925 22:05:44.721259 13139 consensus_queue.cc:260] T e05cd5545ef94b4eb2f6138eb01b8b9e P bbe00c6771664c5393295b94a89bc4ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:44.721277 12613 catalog_manager.cc:4841] TS 87968f19e8d8461aa549ca61f020e42f (127.5.0.65:33097): tablet acb132d2a8fb489aa8f207355272ff1a (table test-table [id=63e8904beefe4e3e997e8beff0ed85bf]) successfully deleted
I20260925 22:05:44.721347 12613 catalog_manager.cc:4841] TS b1d47aae2b464377b14cd3ac5e8f1d2c (127.5.0.67:32885): tablet acb132d2a8fb489aa8f207355272ff1a (table test-table [id=63e8904beefe4e3e997e8beff0ed85bf]) successfully deleted
I20260925 22:05:44.721359 13139 ts_tablet_manager.cc:1428] T e05cd5545ef94b4eb2f6138eb01b8b9e P bbe00c6771664c5393295b94a89bc4ae: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260925 22:05:44.721565 13144 tablet_bootstrap.cc:492] T e05cd5545ef94b4eb2f6138eb01b8b9e P b1d47aae2b464377b14cd3ac5e8f1d2c: No bootstrap required, opened a new log
I20260925 22:05:44.721611 13140 tablet_bootstrap.cc:492] T e05cd5545ef94b4eb2f6138eb01b8b9e P 87968f19e8d8461aa549ca61f020e42f: No bootstrap required, opened a new log
I20260925 22:05:44.721611 13144 ts_tablet_manager.cc:1397] T e05cd5545ef94b4eb2f6138eb01b8b9e P b1d47aae2b464377b14cd3ac5e8f1d2c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 22:05:44.721648 13140 ts_tablet_manager.cc:1397] T e05cd5545ef94b4eb2f6138eb01b8b9e P 87968f19e8d8461aa549ca61f020e42f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 22:05:44.721706 13144 raft_consensus.cc:348] T e05cd5545ef94b4eb2f6138eb01b8b9e P b1d47aae2b464377b14cd3ac5e8f1d2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:44.721783 13144 raft_consensus.cc:374] T e05cd5545ef94b4eb2f6138eb01b8b9e P b1d47aae2b464377b14cd3ac5e8f1d2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:44.721752 13140 raft_consensus.cc:348] T e05cd5545ef94b4eb2f6138eb01b8b9e P 87968f19e8d8461aa549ca61f020e42f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:44.721813 13144 raft_consensus.cc:729] T e05cd5545ef94b4eb2f6138eb01b8b9e P b1d47aae2b464377b14cd3ac5e8f1d2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1d47aae2b464377b14cd3ac5e8f1d2c, State: Initialized, Role: FOLLOWER
I20260925 22:05:44.721820 13140 raft_consensus.cc:374] T e05cd5545ef94b4eb2f6138eb01b8b9e P 87968f19e8d8461aa549ca61f020e42f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:44.721843 13140 raft_consensus.cc:729] T e05cd5545ef94b4eb2f6138eb01b8b9e P 87968f19e8d8461aa549ca61f020e42f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87968f19e8d8461aa549ca61f020e42f, State: Initialized, Role: FOLLOWER
I20260925 22:05:44.721855 13144 consensus_queue.cc:260] T e05cd5545ef94b4eb2f6138eb01b8b9e P b1d47aae2b464377b14cd3ac5e8f1d2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:44.721886 13140 consensus_queue.cc:260] T e05cd5545ef94b4eb2f6138eb01b8b9e P 87968f19e8d8461aa549ca61f020e42f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:44.721947 13144 ts_tablet_manager.cc:1428] T e05cd5545ef94b4eb2f6138eb01b8b9e P b1d47aae2b464377b14cd3ac5e8f1d2c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:44.721984 13140 ts_tablet_manager.cc:1428] T e05cd5545ef94b4eb2f6138eb01b8b9e P 87968f19e8d8461aa549ca61f020e42f: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260925 22:05:44.825395 13145 raft_consensus.cc:482] T e05cd5545ef94b4eb2f6138eb01b8b9e P 87968f19e8d8461aa549ca61f020e42f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:05:44.825496 13145 raft_consensus.cc:504] T e05cd5545ef94b4eb2f6138eb01b8b9e P 87968f19e8d8461aa549ca61f020e42f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:44.825711 13145 leader_election.cc:288] T e05cd5545ef94b4eb2f6138eb01b8b9e P 87968f19e8d8461aa549ca61f020e42f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bbe00c6771664c5393295b94a89bc4ae (127.5.0.66:40453), b1d47aae2b464377b14cd3ac5e8f1d2c (127.5.0.67:32885)
I20260925 22:05:44.825949 12908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e05cd5545ef94b4eb2f6138eb01b8b9e" candidate_uuid: "87968f19e8d8461aa549ca61f020e42f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbe00c6771664c5393295b94a89bc4ae" is_pre_election: true
I20260925 22:05:44.825973 13065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e05cd5545ef94b4eb2f6138eb01b8b9e" candidate_uuid: "87968f19e8d8461aa549ca61f020e42f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" is_pre_election: true
I20260925 22:05:44.826051 12908 raft_consensus.cc:2454] T e05cd5545ef94b4eb2f6138eb01b8b9e P bbe00c6771664c5393295b94a89bc4ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87968f19e8d8461aa549ca61f020e42f in term 0.
I20260925 22:05:44.826061 13065 raft_consensus.cc:2454] T e05cd5545ef94b4eb2f6138eb01b8b9e P b1d47aae2b464377b14cd3ac5e8f1d2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87968f19e8d8461aa549ca61f020e42f in term 0.
I20260925 22:05:44.826208 12718 leader_election.cc:302] T e05cd5545ef94b4eb2f6138eb01b8b9e P 87968f19e8d8461aa549ca61f020e42f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87968f19e8d8461aa549ca61f020e42f, bbe00c6771664c5393295b94a89bc4ae; no voters:
I20260925 22:05:44.826304 13145 raft_consensus.cc:2781] T e05cd5545ef94b4eb2f6138eb01b8b9e P 87968f19e8d8461aa549ca61f020e42f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:05:44.826366 13145 raft_consensus.cc:482] T e05cd5545ef94b4eb2f6138eb01b8b9e P 87968f19e8d8461aa549ca61f020e42f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:05:44.826392 13145 raft_consensus.cc:3037] T e05cd5545ef94b4eb2f6138eb01b8b9e P 87968f19e8d8461aa549ca61f020e42f [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:44.827052 13145 raft_consensus.cc:504] T e05cd5545ef94b4eb2f6138eb01b8b9e P 87968f19e8d8461aa549ca61f020e42f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:44.827232 13145 leader_election.cc:288] T e05cd5545ef94b4eb2f6138eb01b8b9e P 87968f19e8d8461aa549ca61f020e42f [CANDIDATE]: Term 1 election: Requested vote from peers bbe00c6771664c5393295b94a89bc4ae (127.5.0.66:40453), b1d47aae2b464377b14cd3ac5e8f1d2c (127.5.0.67:32885)
I20260925 22:05:44.827360 12908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e05cd5545ef94b4eb2f6138eb01b8b9e" candidate_uuid: "87968f19e8d8461aa549ca61f020e42f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbe00c6771664c5393295b94a89bc4ae"
I20260925 22:05:44.827443 12908 raft_consensus.cc:3037] T e05cd5545ef94b4eb2f6138eb01b8b9e P bbe00c6771664c5393295b94a89bc4ae [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:44.827397 13065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e05cd5545ef94b4eb2f6138eb01b8b9e" candidate_uuid: "87968f19e8d8461aa549ca61f020e42f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c"
I20260925 22:05:44.827488 13065 raft_consensus.cc:3037] T e05cd5545ef94b4eb2f6138eb01b8b9e P b1d47aae2b464377b14cd3ac5e8f1d2c [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:44.828130 13065 raft_consensus.cc:2454] T e05cd5545ef94b4eb2f6138eb01b8b9e P b1d47aae2b464377b14cd3ac5e8f1d2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87968f19e8d8461aa549ca61f020e42f in term 1.
I20260925 22:05:44.828133 12908 raft_consensus.cc:2454] T e05cd5545ef94b4eb2f6138eb01b8b9e P bbe00c6771664c5393295b94a89bc4ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87968f19e8d8461aa549ca61f020e42f in term 1.
I20260925 22:05:44.828311 12717 leader_election.cc:302] T e05cd5545ef94b4eb2f6138eb01b8b9e P 87968f19e8d8461aa549ca61f020e42f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87968f19e8d8461aa549ca61f020e42f, b1d47aae2b464377b14cd3ac5e8f1d2c; no voters:
I20260925 22:05:44.828416 13145 raft_consensus.cc:2781] T e05cd5545ef94b4eb2f6138eb01b8b9e P 87968f19e8d8461aa549ca61f020e42f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:44.828500 13145 raft_consensus.cc:686] T e05cd5545ef94b4eb2f6138eb01b8b9e P 87968f19e8d8461aa549ca61f020e42f [term 1 LEADER]: Becoming Leader. State: Replica: 87968f19e8d8461aa549ca61f020e42f, State: Running, Role: LEADER
I20260925 22:05:44.828554 13145 consensus_queue.cc:237] T e05cd5545ef94b4eb2f6138eb01b8b9e P 87968f19e8d8461aa549ca61f020e42f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } }
I20260925 22:05:44.829133 12626 catalog_manager.cc:5495] T e05cd5545ef94b4eb2f6138eb01b8b9e P 87968f19e8d8461aa549ca61f020e42f reported cstate change: term changed from 0 to 1, leader changed from <none> to 87968f19e8d8461aa549ca61f020e42f (127.5.0.65). New cstate: current_term: 1 leader_uuid: "87968f19e8d8461aa549ca61f020e42f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87968f19e8d8461aa549ca61f020e42f" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 33097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 } health_report { overall_health: UNKNOWN } } }
I20260925 22:05:44.838896 13138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:44.836707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60016 (local address 127.5.0.65:33097)
0925 22:05:44.836743 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:44.836745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:44.836768 (+ 23us) server_negotiation.cc:407] Connection header received
0925 22:05:44.836843 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:44.836844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:44.836853 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:44.836903 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:05:44.837017 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.837488 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.838212 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.838409 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.838491 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:44.838644 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:05:44.838679 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:05:44.838742 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:44.838793 (+ 51us) server_negotiation.cc:299] Negotiation successful
0925 22:05:44.838815 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 22:05:44.842669 13138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:44.840846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:43827 (local address 127.5.0.65:33097)
0925 22:05:44.840898 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:44.840899 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:44.840907 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:05:44.840941 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:44.840941 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:44.840948 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:44.840994 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:05:44.841063 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.841494 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.842144 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.842316 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.842364 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:44.842477 (+ 113us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:05:44.842498 (+ 21us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:05:44.842534 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:44.842583 (+ 49us) server_negotiation.cc:299] Negotiation successful
0925 22:05:44.842596 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 22:05:44.842715 13207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:44.840789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:33097 (local address 127.5.0.65:43827)
0925 22:05:44.840843 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:44.840854 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:44.840903 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:44.840998 (+ 95us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:44.840998 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:44.840999 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 22:05:44.841031 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:44.841032 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.841547 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.841548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:44.842077 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:44.842078 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:44.842375 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:44.842375 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:44.842410 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:44.842415 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 22:05:44.842551 (+ 136us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 22:05:44.842551 (+ 0us) client_negotiation.cc:769] Sending connection context
0925 22:05:44.842583 (+ 32us) client_negotiation.cc:240] Negotiation successful
0925 22:05:44.842615 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260925 22:05:44.844445 13065 raft_consensus.cc:1261] T e05cd5545ef94b4eb2f6138eb01b8b9e P b1d47aae2b464377b14cd3ac5e8f1d2c [term 1 FOLLOWER]: Refusing update from remote peer 87968f19e8d8461aa549ca61f020e42f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:05:44.844462 12908 raft_consensus.cc:1261] T e05cd5545ef94b4eb2f6138eb01b8b9e P bbe00c6771664c5393295b94a89bc4ae [term 1 FOLLOWER]: Refusing update from remote peer 87968f19e8d8461aa549ca61f020e42f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:05:44.844656 13145 consensus_queue.cc:1035] T e05cd5545ef94b4eb2f6138eb01b8b9e P 87968f19e8d8461aa549ca61f020e42f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbe00c6771664c5393295b94a89bc4ae" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 40453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:05:44.844750 13200 consensus_queue.cc:1035] T e05cd5545ef94b4eb2f6138eb01b8b9e P 87968f19e8d8461aa549ca61f020e42f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1d47aae2b464377b14cd3ac5e8f1d2c" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 32885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:05:44.854465 12627 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:05:44.854562 12627 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:05:44.855181 12627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f5a748e94d32429ca130f24868a54d8c: Sending DeleteTablet for 3 replicas of tablet e05cd5545ef94b4eb2f6138eb01b8b9e
I20260925 22:05:44.855410 12905 tablet_service.cc:1514] Processing DeleteTablet for tablet e05cd5545ef94b4eb2f6138eb01b8b9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:44 UTC) from {username='oryx', principal='oryx/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:36776
I20260925 22:05:44.855433 13045 tablet_service.cc:1514] Processing DeleteTablet for tablet e05cd5545ef94b4eb2f6138eb01b8b9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:44 UTC) from {username='oryx', principal='oryx/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:54710
I20260925 22:05:44.855476 12765 tablet_service.cc:1514] Processing DeleteTablet for tablet e05cd5545ef94b4eb2f6138eb01b8b9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:05:44 UTC) from {username='oryx', principal='oryx/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:60000
I20260925 22:05:44.855605 13212 tablet_replica.cc:330] T e05cd5545ef94b4eb2f6138eb01b8b9e P 87968f19e8d8461aa549ca61f020e42f: stopping tablet replica
I20260925 22:05:44.855646 13211 tablet_replica.cc:330] T e05cd5545ef94b4eb2f6138eb01b8b9e P b1d47aae2b464377b14cd3ac5e8f1d2c: stopping tablet replica
I20260925 22:05:44.855659 13212 raft_consensus.cc:2229] T e05cd5545ef94b4eb2f6138eb01b8b9e P 87968f19e8d8461aa549ca61f020e42f [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:05:44.855703 13211 raft_consensus.cc:2229] T e05cd5545ef94b4eb2f6138eb01b8b9e P b1d47aae2b464377b14cd3ac5e8f1d2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:05:44.855741 13211 raft_consensus.cc:2258] T e05cd5545ef94b4eb2f6138eb01b8b9e P b1d47aae2b464377b14cd3ac5e8f1d2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:44.855741 13212 raft_consensus.cc:2258] T e05cd5545ef94b4eb2f6138eb01b8b9e P 87968f19e8d8461aa549ca61f020e42f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:44.856149 13214 tablet_replica.cc:330] T e05cd5545ef94b4eb2f6138eb01b8b9e P bbe00c6771664c5393295b94a89bc4ae: stopping tablet replica
I20260925 22:05:44.856223 13214 raft_consensus.cc:2229] T e05cd5545ef94b4eb2f6138eb01b8b9e P bbe00c6771664c5393295b94a89bc4ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:05:44.856258 13214 raft_consensus.cc:2258] T e05cd5545ef94b4eb2f6138eb01b8b9e P bbe00c6771664c5393295b94a89bc4ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:05:44.856317 13211 ts_tablet_manager.cc:1905] T e05cd5545ef94b4eb2f6138eb01b8b9e P b1d47aae2b464377b14cd3ac5e8f1d2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:44.856523 13212 ts_tablet_manager.cc:1905] T e05cd5545ef94b4eb2f6138eb01b8b9e P 87968f19e8d8461aa549ca61f020e42f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:44.856544 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 12700
I20260925 22:05:44.858672 13214 ts_tablet_manager.cc:1905] T e05cd5545ef94b4eb2f6138eb01b8b9e P bbe00c6771664c5393295b94a89bc4ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:05:44.858992 13211 ts_tablet_manager.cc:1918] T e05cd5545ef94b4eb2f6138eb01b8b9e P b1d47aae2b464377b14cd3ac5e8f1d2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 22:05:44.859051 13211 log.cc:1196] T e05cd5545ef94b4eb2f6138eb01b8b9e P b1d47aae2b464377b14cd3ac5e8f1d2c: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-2/wal/wals/e05cd5545ef94b4eb2f6138eb01b8b9e
I20260925 22:05:44.859269 13211 ts_tablet_manager.cc:1939] T e05cd5545ef94b4eb2f6138eb01b8b9e P b1d47aae2b464377b14cd3ac5e8f1d2c: Deleting consensus metadata
I20260925 22:05:44.859647 12613 catalog_manager.cc:4841] TS b1d47aae2b464377b14cd3ac5e8f1d2c (127.5.0.67:32885): tablet e05cd5545ef94b4eb2f6138eb01b8b9e (table test-table [id=62b917d508ac4caaac996c3b30abac7a]) successfully deleted
I20260925 22:05:44.860385 13214 ts_tablet_manager.cc:1918] T e05cd5545ef94b4eb2f6138eb01b8b9e P bbe00c6771664c5393295b94a89bc4ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 22:05:44.860450 13214 log.cc:1196] T e05cd5545ef94b4eb2f6138eb01b8b9e P bbe00c6771664c5393295b94a89bc4ae: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373908987421-5121-0/minicluster-data/ts-1/wal/wals/e05cd5545ef94b4eb2f6138eb01b8b9e
I20260925 22:05:44.860651 13214 ts_tablet_manager.cc:1939] T e05cd5545ef94b4eb2f6138eb01b8b9e P bbe00c6771664c5393295b94a89bc4ae: Deleting consensus metadata
I20260925 22:05:44.860984 12614 catalog_manager.cc:4841] TS bbe00c6771664c5393295b94a89bc4ae (127.5.0.66:40453): tablet e05cd5545ef94b4eb2f6138eb01b8b9e (table test-table [id=62b917d508ac4caaac996c3b30abac7a]) successfully deleted
W20260925 22:05:44.863997 12613 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.0.65:33097 (error 108)
I20260925 22:05:44.864244 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 12840
I20260925 22:05:44.865075 13245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:44.864403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:33097 (local address 127.0.0.1:60018)
0925 22:05:44.865002 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:44.865042 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.65:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":57,"threads_started":1}
W20260925 22:05:44.865219 12613 catalog_manager.cc:4584] TS 87968f19e8d8461aa549ca61f020e42f (127.5.0.65:33097): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e05cd5545ef94b4eb2f6138eb01b8b9e: Network error: Client connection negotiation failed: client connection to 127.5.0.65:33097: connect: Connection refused (error 111)
I20260925 22:05:44.870760 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 12980
I20260925 22:05:44.876618 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 12596
2026-09-25T22:05:44Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3151 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:05:44 dist-test-slave-7mqr krb5kdc[13248](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:05:44 dist-test-slave-7mqr krb5kdc[13248](info): set up 2 sockets
Sep 25 22:05:44 dist-test-slave-7mqr krb5kdc[13248](info): commencing operation
krb5kdc: starting...
W20260925 22:05:45.984452 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:05:46 dist-test-slave-7mqr krb5kdc[13248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:05:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:05:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.0.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
I20260925 22:05:46.100765 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:33655
--webserver_interface=127.5.0.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:34675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:33655
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:46.181342 13261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:46.181655 13261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:46.181694 13261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:46.183707 13261 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:05:46.183773 13261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:46.183796 13261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:46.183818 13261 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:05:46.183846 13261 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:05:46.186357 13261 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:34675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:33655
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:33655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:46.186646 13261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:46.186902 13261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:46.191603 13269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:46.191617 13266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:46.191622 13267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:46.191705 13261 server_base.cc:1034] running on GCE node
I20260925 22:05:46.192044 13261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:46.192319 13261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:46.193454 13261 hybrid_clock.cc:648] HybridClock initialized: now 1790373946193444 us; error 29 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-25T22:05:46Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1320 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:05:46 dist-test-slave-7mqr krb5kdc[13274](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:05:46 dist-test-slave-7mqr krb5kdc[13274](info): set up 2 sockets
Sep 25 22:05:46 dist-test-slave-7mqr krb5kdc[13274](info): commencing operation
krb5kdc: starting...
W20260925 22:05:47.220564 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:05:47 dist-test-slave-7mqr krb5kdc[13274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:05:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:05:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.0.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
I20260925 22:05:47.336827 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:38597
--webserver_interface=127.5.0.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:35089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:38597
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:47.416332 13287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:47.416577 13287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:47.416610 13287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:47.417819 13287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:05:47.417884 13287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:47.417909 13287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:47.417941 13287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:05:47.417968 13287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:05:47.419474 13287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:35089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:38597
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:38597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:47.419754 13287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:47.419994 13287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:47.424738 13295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:47.424746 13293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:47.424746 13292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:47.424949 13287 server_base.cc:1034] running on GCE node
I20260925 22:05:47.425225 13287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:47.425472 13287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:47.426690 13287 hybrid_clock.cc:648] HybridClock initialized: now 1790373947426671 us; error 45 us; skew 500 ppm
Sep 25 22:05:47 dist-test-slave-7mqr krb5kdc[13274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373947, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:47.436560 13287 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:05:47.437076 13287 webserver.cc:460] Webserver started at http://127.5.0.126:44851/ using document root <none> and password file <none>
I20260925 22:05:47.437323 13287 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:47.437392 13287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:47.437577 13287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:47.438664 13287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "3444a9bc0afc4a9eabc091ed4c454bf0"
format_stamp: "Formatted at 2026-09-25 22:05:47 on dist-test-slave-7mqr"
server_key: "d556cabd5436b21576f20959571ca6ac"
server_key_iv: "5ed059a2342b310f6c61688643257016"
server_key_version: "encryptionkey@0"
I20260925 22:05:47.439023 13287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "3444a9bc0afc4a9eabc091ed4c454bf0"
format_stamp: "Formatted at 2026-09-25 22:05:47 on dist-test-slave-7mqr"
server_key: "d556cabd5436b21576f20959571ca6ac"
server_key_iv: "5ed059a2342b310f6c61688643257016"
server_key_version: "encryptionkey@0"
I20260925 22:05:47.440449 13287 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:05:47.441210 13302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:47.441411 13287 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:05:47.441569 13287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "3444a9bc0afc4a9eabc091ed4c454bf0"
format_stamp: "Formatted at 2026-09-25 22:05:47 on dist-test-slave-7mqr"
server_key: "d556cabd5436b21576f20959571ca6ac"
server_key_iv: "5ed059a2342b310f6c61688643257016"
server_key_version: "encryptionkey@0"
I20260925 22:05:47.441661 13287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:47.466431 13287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:47.468421 13287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:47.468631 13287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:47.472906 13287 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:38597
I20260925 22:05:47.472946 13354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:38597 every 8 connection(s)
I20260925 22:05:47.473299 13287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:05:47.474124 13355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:05:47.476347 13355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0: Bootstrap starting.
I20260925 22:05:47.476922 13355 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0: Neither blocks nor log segments found. Creating new log.
I20260925 22:05:47.477211 13355 log.cc:824] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:47.477892 13355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0: No bootstrap required, opened a new log
I20260925 22:05:47.479411 13355 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3444a9bc0afc4a9eabc091ed4c454bf0" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 38597 } }
I20260925 22:05:47.479543 13355 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:05:47.479565 13355 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3444a9bc0afc4a9eabc091ed4c454bf0, State: Initialized, Role: FOLLOWER
I20260925 22:05:47.479728 13355 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3444a9bc0afc4a9eabc091ed4c454bf0" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 38597 } }
I20260925 22:05:47.479808 13355 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:05:47.479853 13355 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:05:47.479921 13355 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:05:47.480412 13355 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3444a9bc0afc4a9eabc091ed4c454bf0" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 38597 } }
I20260925 22:05:47.480517 13355 leader_election.cc:302] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3444a9bc0afc4a9eabc091ed4c454bf0; no voters:
I20260925 22:05:47.480690 13355 leader_election.cc:288] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:05:47.480749 13358 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:05:47.480945 13355 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:05:47.481194 13358 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [term 1 LEADER]: Becoming Leader. State: Replica: 3444a9bc0afc4a9eabc091ed4c454bf0, State: Running, Role: LEADER
I20260925 22:05:47.481289 13358 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3444a9bc0afc4a9eabc091ed4c454bf0" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 38597 } }
I20260925 22:05:47.481757 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 13287
I20260925 22:05:47.481720 13360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3444a9bc0afc4a9eabc091ed4c454bf0. Latest consensus state: current_term: 1 leader_uuid: "3444a9bc0afc4a9eabc091ed4c454bf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3444a9bc0afc4a9eabc091ed4c454bf0" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 38597 } } }
I20260925 22:05:47.481740 13359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3444a9bc0afc4a9eabc091ed4c454bf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3444a9bc0afc4a9eabc091ed4c454bf0" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 38597 } } }
I20260925 22:05:47.481813 13360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:47.481841 13359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:47.481873 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:05:47.482007 13367 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:05:47.482106 5121 external_mini_cluster.cc:1428] Setting key ff7ce0977e1c983f5cd823737d368c86
I20260925 22:05:47.482539 13367 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:05:47.484237 13367 catalog_manager.cc:1347] Generated new cluster ID: a0bbd05292bd433ead9a817a5345113d
I20260925 22:05:47.484311 13367 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:05:47 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:47.491245 13367 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:05:47.491959 13367 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:05:47.494912 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:47.482580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37878 (local address 127.5.0.126:38597)
0925 22:05:47.483317 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:47.483326 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:47.483748 (+ 422us) server_negotiation.cc:407] Connection header received
0925 22:05:47.483917 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:47.483924 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:47.483982 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:47.484119 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:47.485119 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:47.486523 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:47.487597 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:47.488210 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:47.488302 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:47.491240 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:47.491243 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:47.491244 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:47.491256 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:47.493283 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:47.493692 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:47.493695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:47.493695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:47.493795 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:47.494011 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:47.494012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:47.494012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:47.494284 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:47.494421 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:47.494532 (+ 111us) server_negotiation.cc:299] Negotiation successful
0925 22:05:47.494667 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":72,"threads_started":1}
I20260925 22:05:47.502542 13367 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0: Generated new TSK 0
I20260925 22:05:47.502729 13367 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.0.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
I20260925 22:05:47.526768 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.65:0
--local_ip_for_outbound_sockets=127.5.0.65
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:38597
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:35089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:47.606976 13381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:47.607208 13381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:47.607249 13381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:47.608495 13381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:47.608557 13381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:47.608603 13381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.65
I20260925 22:05:47.610065 13381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:35089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:38597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.65
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:47.610332 13381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:47.610555 13381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:47.615839 13386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:47.615852 13387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:47.615839 13389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:47.616043 13381 server_base.cc:1034] running on GCE node
I20260925 22:05:47.616314 13381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:47.616586 13381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:47.617794 13381 hybrid_clock.cc:648] HybridClock initialized: now 1790373947617789 us; error 39 us; skew 500 ppm
Sep 25 22:05:47 dist-test-slave-7mqr krb5kdc[13274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373947, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:47.620332 13381 init.cc:377] Logged in from keytab as kudu/127.5.0.65@KRBTEST.COM (short username kudu)
I20260925 22:05:47.620761 13381 webserver.cc:460] Webserver started at http://127.5.0.65:45833/ using document root <none> and password file <none>
I20260925 22:05:47.621001 13381 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:47.621075 13381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:47.621294 13381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:47.622294 13381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-0/data/instance:
uuid: "82fca1cff13f4034bdb4ef3057d66fc4"
format_stamp: "Formatted at 2026-09-25 22:05:47 on dist-test-slave-7mqr"
server_key: "f8c7f35b73b1d35b38cad938ff591105"
server_key_iv: "8b958817c6a76e2d997a4df5f6826f40"
server_key_version: "encryptionkey@0"
I20260925 22:05:47.622588 13381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance:
uuid: "82fca1cff13f4034bdb4ef3057d66fc4"
format_stamp: "Formatted at 2026-09-25 22:05:47 on dist-test-slave-7mqr"
server_key: "f8c7f35b73b1d35b38cad938ff591105"
server_key_iv: "8b958817c6a76e2d997a4df5f6826f40"
server_key_version: "encryptionkey@0"
I20260925 22:05:47.623961 13381 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:05:47.624711 13396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:47.624887 13381 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:05:47.624969 13381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-0/wal
uuid: "82fca1cff13f4034bdb4ef3057d66fc4"
format_stamp: "Formatted at 2026-09-25 22:05:47 on dist-test-slave-7mqr"
server_key: "f8c7f35b73b1d35b38cad938ff591105"
server_key_iv: "8b958817c6a76e2d997a4df5f6826f40"
server_key_version: "encryptionkey@0"
I20260925 22:05:47.625025 13381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:47.634739 13381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:47.636612 13381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:47.636737 13381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:47.636907 13381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:47.637207 13381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:47.637239 13381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:47.637310 13381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:47.637337 13381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:47.642725 13381 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.65:33749
I20260925 22:05:47.642783 13509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.65:33749 every 8 connection(s)
I20260925 22:05:47.643108 13381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
Sep 25 22:05:47 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373947, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:47.651736 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 13381
I20260925 22:05:47.651871 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance
I20260925 22:05:47.652129 5121 external_mini_cluster.cc:1428] Setting key d2edd971599bf97112e0f312d5733b2f
I20260925 22:05:47.652348 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:47.643926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:56783 (local address 127.5.0.126:38597)
0925 22:05:47.643976 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:47.643978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:47.644194 (+ 216us) server_negotiation.cc:407] Connection header received
0925 22:05:47.644760 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:47.644762 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:47.644774 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:47.644825 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:47.645809 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:47.646322 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:47.647337 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:47.647557 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:47.647634 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:47.649724 (+ 2090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:47.649725 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:47.649726 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:47.649733 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:47.650903 (+ 1170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:47.651308 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:47.651309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:47.651310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:47.651372 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:47.651620 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:47.651621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:47.651621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:47.651738 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:47.651815 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:47.652160 (+ 345us) server_negotiation.cc:299] Negotiation successful
0925 22:05:47.652259 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 22:05:47.652524 13512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:47.643882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.65:56783)
0925 22:05:47.644068 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:47.644103 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:47.644634 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:47.644894 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:47.644898 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:47.645518 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:47.645714 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:47.645719 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:47.646421 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:47.646422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:47.647233 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:47.647235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:47.647649 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:47.647650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:47.647781 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:47.648288 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:47.648296 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:47.649595 (+ 1299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:47.650999 (+ 1404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:47.651005 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:47.651008 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:47.651229 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:47.651439 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:47.651440 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:47.651440 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:47.651551 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:47.651805 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:47.651811 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:47.652011 (+ 200us) client_negotiation.cc:769] Sending connection context
0925 22:05:47.652140 (+ 129us) client_negotiation.cc:240] Negotiation successful
0925 22:05:47.652291 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":82,"threads_started":1}
I20260925 22:05:47.653151 13510 heartbeater.cc:347] Connected to a master server at 127.5.0.126:38597
I20260925 22:05:47.653231 13510 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:47.653421 13510 heartbeater.cc:511] Master 127.5.0.126:38597 requested a full tablet report, sending...
I20260925 22:05:47.653918 13319 ts_manager.cc:194] Registered new tserver with Master: 82fca1cff13f4034bdb4ef3057d66fc4 (127.5.0.65:33749)
I20260925 22:05:47.654522 13319 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.65@KRBTEST.COM'} at 127.5.0.65:56783
WARNING: no policy specified for kudu/127.5.0.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
I20260925 22:05:47.674739 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.66:0
--local_ip_for_outbound_sockets=127.5.0.66
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:38597
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:35089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:47.754737 13515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:47.754977 13515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:47.755014 13515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:47.756268 13515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:47.756327 13515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:47.756371 13515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.66
I20260925 22:05:47.757797 13515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:35089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:38597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.66
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:47.758056 13515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:47.758285 13515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:47.763048 13520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:47.763108 13521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:47.763049 13523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:47.763273 13515 server_base.cc:1034] running on GCE node
I20260925 22:05:47.763511 13515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:47.763747 13515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:47.764899 13515 hybrid_clock.cc:648] HybridClock initialized: now 1790373947764879 us; error 30 us; skew 500 ppm
Sep 25 22:05:47 dist-test-slave-7mqr krb5kdc[13274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373947, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:47.767270 13515 init.cc:377] Logged in from keytab as kudu/127.5.0.66@KRBTEST.COM (short username kudu)
I20260925 22:05:47.767700 13515 webserver.cc:460] Webserver started at http://127.5.0.66:33657/ using document root <none> and password file <none>
I20260925 22:05:47.767925 13515 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:47.767979 13515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:47.768069 13515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:47.769043 13515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-1/data/instance:
uuid: "ecd0620b63eb473b9f4b9bbc689d5eba"
format_stamp: "Formatted at 2026-09-25 22:05:47 on dist-test-slave-7mqr"
server_key: "9e72c4f8ba9adfee757ab85425958e8d"
server_key_iv: "bc7c4dddf24fe353db8544dcc5c5dbd2"
server_key_version: "encryptionkey@0"
I20260925 22:05:47.769332 13515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance:
uuid: "ecd0620b63eb473b9f4b9bbc689d5eba"
format_stamp: "Formatted at 2026-09-25 22:05:47 on dist-test-slave-7mqr"
server_key: "9e72c4f8ba9adfee757ab85425958e8d"
server_key_iv: "bc7c4dddf24fe353db8544dcc5c5dbd2"
server_key_version: "encryptionkey@0"
I20260925 22:05:47.770570 13515 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:05:47.771245 13530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:47.771397 13515 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:05:47.771471 13515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-1/wal
uuid: "ecd0620b63eb473b9f4b9bbc689d5eba"
format_stamp: "Formatted at 2026-09-25 22:05:47 on dist-test-slave-7mqr"
server_key: "9e72c4f8ba9adfee757ab85425958e8d"
server_key_iv: "bc7c4dddf24fe353db8544dcc5c5dbd2"
server_key_version: "encryptionkey@0"
I20260925 22:05:47.771522 13515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:47.787783 13515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:47.790211 13515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:47.790328 13515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:47.790511 13515 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:47.790838 13515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:47.790874 13515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:47.791002 13515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:47.791028 13515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:47.796680 13515 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.66:42785
I20260925 22:05:47.796762 13643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.66:42785 every 8 connection(s)
I20260925 22:05:47.797102 13515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
I20260925 22:05:47.799319 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 13515
I20260925 22:05:47.799438 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance
I20260925 22:05:47.799644 5121 external_mini_cluster.cc:1428] Setting key b458eed290b0f5c45f50927e0fbfa4a7
Sep 25 22:05:47 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373947, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:47.807828 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:47.797851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:53091 (local address 127.5.0.126:38597)
0925 22:05:47.797898 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:47.797900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:47.798097 (+ 197us) server_negotiation.cc:407] Connection header received
0925 22:05:47.798616 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:47.798617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:47.798633 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:47.798703 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:47.799981 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:47.800586 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:47.801606 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:47.801814 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:47.801902 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:47.804516 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:47.804517 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:47.804517 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:47.804524 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:47.806132 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:47.806728 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:47.806729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:47.806729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:47.806785 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:47.807103 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:47.807104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:47.807104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:47.807224 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:47.807275 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:47.807686 (+ 411us) server_negotiation.cc:299] Negotiation successful
0925 22:05:47.807758 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 22:05:47.808276 13646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:47.797817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.66:53091)
0925 22:05:47.797974 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:47.798008 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:47.798509 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:47.798783 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:47.798786 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:47.799530 (+ 744us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:47.799838 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:47.799844 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:47.800674 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:47.800675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:47.801490 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:47.801492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:47.801913 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:47.801914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:47.802051 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:47.802610 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:47.802619 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:47.804364 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:47.806227 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:47.806234 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:47.806237 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:47.806629 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:47.806862 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:47.806863 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:47.806864 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:47.807026 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:47.807296 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:47.807303 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:47.807508 (+ 205us) client_negotiation.cc:769] Sending connection context
0925 22:05:47.807669 (+ 161us) client_negotiation.cc:240] Negotiation successful
0925 22:05:47.807965 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
I20260925 22:05:47.808851 13644 heartbeater.cc:347] Connected to a master server at 127.5.0.126:38597
I20260925 22:05:47.808959 13644 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.5.0.67@KRBTEST.COM; defaulting to no policy
I20260925 22:05:47.809188 13644 heartbeater.cc:511] Master 127.5.0.126:38597 requested a full tablet report, sending...
I20260925 22:05:47.809590 13319 ts_manager.cc:194] Registered new tserver with Master: ecd0620b63eb473b9f4b9bbc689d5eba (127.5.0.66:42785)
I20260925 22:05:47.809981 13319 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.66@KRBTEST.COM'} at 127.5.0.66:53091
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
I20260925 22:05:47.826122 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.67:0
--local_ip_for_outbound_sockets=127.5.0.67
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:38597
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:35089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:47.907143 13649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:47.907364 13649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:47.907387 13649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:47.908604 13649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:47.908651 13649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:47.908679 13649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.67
I20260925 22:05:47.910130 13649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:35089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:38597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.67
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:47.910373 13649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:47.910688 13649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:47.916224 13657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:47.916203 13655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:47.916242 13649 server_base.cc:1034] running on GCE node
W20260925 22:05:47.916199 13654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:47.916589 13649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:47.916874 13649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:47.918021 13649 hybrid_clock.cc:648] HybridClock initialized: now 1790373947918012 us; error 33 us; skew 500 ppm
Sep 25 22:05:47 dist-test-slave-7mqr krb5kdc[13274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373947, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:47.920527 13649 init.cc:377] Logged in from keytab as kudu/127.5.0.67@KRBTEST.COM (short username kudu)
I20260925 22:05:47.921041 13649 webserver.cc:460] Webserver started at http://127.5.0.67:43279/ using document root <none> and password file <none>
I20260925 22:05:47.921268 13649 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:47.921334 13649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:47.921490 13649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:05:47.922528 13649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-2/data/instance:
uuid: "8213f62c97ae4af99d96e67972042bf5"
format_stamp: "Formatted at 2026-09-25 22:05:47 on dist-test-slave-7mqr"
server_key: "e47a39a8f59ad259cf4b17bc472293b7"
server_key_iv: "c8a31e86bd3288fee5b1069ef0194ec8"
server_key_version: "encryptionkey@0"
I20260925 22:05:47.922899 13649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance:
uuid: "8213f62c97ae4af99d96e67972042bf5"
format_stamp: "Formatted at 2026-09-25 22:05:47 on dist-test-slave-7mqr"
server_key: "e47a39a8f59ad259cf4b17bc472293b7"
server_key_iv: "c8a31e86bd3288fee5b1069ef0194ec8"
server_key_version: "encryptionkey@0"
I20260925 22:05:47.924248 13649 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:05:47.924960 13664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:47.925129 13649 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:05:47.925230 13649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-2/wal
uuid: "8213f62c97ae4af99d96e67972042bf5"
format_stamp: "Formatted at 2026-09-25 22:05:47 on dist-test-slave-7mqr"
server_key: "e47a39a8f59ad259cf4b17bc472293b7"
server_key_iv: "c8a31e86bd3288fee5b1069ef0194ec8"
server_key_version: "encryptionkey@0"
I20260925 22:05:47.925310 13649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:47.932888 13649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:47.935108 13649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:47.935268 13649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:47.935487 13649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:47.935786 13649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:47.935838 13649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:47.935879 13649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:47.935935 13649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:47.941817 13649 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.67:38355
I20260925 22:05:47.941857 13777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.67:38355 every 8 connection(s)
I20260925 22:05:47.942198 13649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
Sep 25 22:05:47 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373947, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:05:47.951280 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 13649
I20260925 22:05:47.951396 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance
I20260925 22:05:47.951596 5121 external_mini_cluster.cc:1428] Setting key ce501382dfb0f873e5613d966d08b99d
I20260925 22:05:47.951927 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:47.942909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:51115 (local address 127.5.0.126:38597)
0925 22:05:47.942990 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:47.942993 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:47.943222 (+ 229us) server_negotiation.cc:407] Connection header received
0925 22:05:47.943650 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:47.943651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:47.943661 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:47.943719 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:47.945081 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:47.945588 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:47.946618 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:47.946825 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:47.946909 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:47.949203 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:47.949205 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:47.949205 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:47.949211 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:47.950423 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:47.950833 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:47.950834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:47.950835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:47.950883 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:47.951101 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:47.951102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:47.951102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:47.951248 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:47.951321 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:47.951696 (+ 375us) server_negotiation.cc:299] Negotiation successful
0925 22:05:47.951783 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 22:05:47.952109 13780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:47.942894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.67:51115)
0925 22:05:47.943083 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:47.943117 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:47.943568 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:47.943772 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:47.943775 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:47.944698 (+ 923us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:47.944994 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:47.945001 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:47.945692 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:47.945693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:47.946508 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:47.946511 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:47.946919 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:47.946921 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:47.947052 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:47.947608 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:47.947617 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:47.949119 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:05:47.950537 (+ 1418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:47.950543 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:47.950544 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:47.950767 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:47.950940 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:05:47.950941 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:05:47.950941 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:05:47.951039 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:05:47.951316 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:05:47.951320 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:05:47.951541 (+ 221us) client_negotiation.cc:769] Sending connection context
0925 22:05:47.951633 (+ 92us) client_negotiation.cc:240] Negotiation successful
0925 22:05:47.951816 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":110,"threads_started":1}
I20260925 22:05:47.952600 13778 heartbeater.cc:347] Connected to a master server at 127.5.0.126:38597
I20260925 22:05:47.952701 13778 heartbeater.cc:464] Registering TS with master...
I20260925 22:05:47.952886 13778 heartbeater.cc:511] Master 127.5.0.126:38597 requested a full tablet report, sending...
I20260925 22:05:47.953230 13319 ts_manager.cc:194] Registered new tserver with Master: 8213f62c97ae4af99d96e67972042bf5 (127.5.0.67:38355)
I20260925 22:05:47.953552 13319 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.67@KRBTEST.COM'} at 127.5.0.67:51115
I20260925 22:05:47.963615 5121 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.5.0.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.0.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.0.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.65.keytab.
Entry for principal oryx/127.5.0.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.65.keytab.
WARNING: no policy specified for oryx/127.5.0.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.0.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.0.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.66.keytab.
Entry for principal oryx/127.5.0.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.66.keytab.
WARNING: no policy specified for oryx/127.5.0.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.0.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.0.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.67.keytab.
Entry for principal oryx/127.5.0.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.67.keytab.
I20260925 22:05:48.032097 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 13381
I20260925 22:05:48.038762 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 13515
I20260925 22:05:48.044517 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 13649
I20260925 22:05:48.049125 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 13287
I20260925 22:05:48.053944 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:38597
--webserver_interface=127.5.0.126
--webserver_port=44851
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:35089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:38597
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:05:48.134639 13787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:48.134853 13787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:48.134876 13787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:48.136071 13787 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:05:48.136111 13787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:48.136125 13787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:48.136138 13787 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:05:48.136190 13787 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:05:48.137655 13787 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:35089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:38597
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:38597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=44851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:48.137900 13787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:48.138134 13787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 22:05:48.142850 13787 server_base.cc:1034] running on GCE node
W20260925 22:05:48.142789 13792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:48.142791 13795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:48.142835 13793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:48.143273 13787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:48.143558 13787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:48.144726 13787 hybrid_clock.cc:648] HybridClock initialized: now 1790373948144707 us; error 27 us; skew 500 ppm
Sep 25 22:05:48 dist-test-slave-7mqr krb5kdc[13274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373948, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:48.147146 13787 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:05:48.147591 13787 webserver.cc:460] Webserver started at http://127.5.0.126:44851/ using document root <none> and password file <none>
I20260925 22:05:48.147785 13787 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:48.147836 13787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:48.149144 13787 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:05:48.149775 13802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:48.149938 13787 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:05:48.150017 13787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "3444a9bc0afc4a9eabc091ed4c454bf0"
format_stamp: "Formatted at 2026-09-25 22:05:47 on dist-test-slave-7mqr"
server_key: "d556cabd5436b21576f20959571ca6ac"
server_key_iv: "5ed059a2342b310f6c61688643257016"
server_key_version: "encryptionkey@0"
I20260925 22:05:48.150321 13787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:48.176945 13787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:48.179191 13787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:48.179296 13787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:48.183003 13787 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:38597
I20260925 22:05:48.183055 13854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:38597 every 8 connection(s)
I20260925 22:05:48.183432 13787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:05:48.184878 13855 sys_catalog.cc:263] Verifying existing consensus state
I20260925 22:05:48.185505 13855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0: Bootstrap starting.
I20260925 22:05:48.188649 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 13787
I20260925 22:05:48.188984 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.65:33749
--local_ip_for_outbound_sockets=127.5.0.65
--tserver_master_addrs=127.5.0.126:38597
--webserver_port=45833
--webserver_interface=127.5.0.65
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:35089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.0.65
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:05:48.193818 13855 log.cc:824] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0: Log is configured to *not* fsync() on all Append() calls
I20260925 22:05:48.195429 13855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 22:05:48.195721 13855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0: Bootstrap complete.
I20260925 22:05:48.197942 13855 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3444a9bc0afc4a9eabc091ed4c454bf0" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 38597 } }
I20260925 22:05:48.198161 13855 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3444a9bc0afc4a9eabc091ed4c454bf0, State: Initialized, Role: FOLLOWER
I20260925 22:05:48.198324 13855 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3444a9bc0afc4a9eabc091ed4c454bf0" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 38597 } }
I20260925 22:05:48.198391 13855 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:05:48.198439 13855 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:05:48.198485 13855 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [term 1 FOLLOWER]: Advancing to term 2
I20260925 22:05:48.199433 13855 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3444a9bc0afc4a9eabc091ed4c454bf0" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 38597 } }
I20260925 22:05:48.199513 13855 leader_election.cc:302] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3444a9bc0afc4a9eabc091ed4c454bf0; no voters:
I20260925 22:05:48.199720 13855 leader_election.cc:288] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 22:05:48.199755 13859 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 22:05:48.199869 13859 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [term 2 LEADER]: Becoming Leader. State: Replica: 3444a9bc0afc4a9eabc091ed4c454bf0, State: Running, Role: LEADER
I20260925 22:05:48.199966 13859 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3444a9bc0afc4a9eabc091ed4c454bf0" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 38597 } }
I20260925 22:05:48.199991 13855 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:05:48.200223 13861 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3444a9bc0afc4a9eabc091ed4c454bf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3444a9bc0afc4a9eabc091ed4c454bf0" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 38597 } } }
I20260925 22:05:48.200299 13861 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:48.200503 13860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3444a9bc0afc4a9eabc091ed4c454bf0. Latest consensus state: current_term: 2 leader_uuid: "3444a9bc0afc4a9eabc091ed4c454bf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3444a9bc0afc4a9eabc091ed4c454bf0" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 38597 } } }
I20260925 22:05:48.200573 13860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0 [sys.catalog]: This master's current role is: LEADER
I20260925 22:05:48.200635 13863 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:05:48.201131 13863 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:05:48.201260 13863 catalog_manager.cc:1259] Loaded cluster ID: a0bbd05292bd433ead9a817a5345113d
I20260925 22:05:48.201299 13863 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:05:48.202478 13863 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:05:48.202659 13863 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3444a9bc0afc4a9eabc091ed4c454bf0: Loaded TSK: 0
I20260925 22:05:48.202863 13863 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 22:05:48.270551 13857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:48.270781 13857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:48.270821 13857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:48.272059 13857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:48.272115 13857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:48.272161 13857 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.65
I20260925 22:05:48.273664 13857 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:35089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.65.keytab
--principal=oryx/127.5.0.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.65:33749
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.0.65
--webserver_port=45833
--tserver_master_addrs=127.5.0.126:38597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13857
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.65
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:48.273918 13857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:48.274138 13857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:48.279968 13882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:48.280052 13857 server_base.cc:1034] running on GCE node
W20260925 22:05:48.280067 13881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:48.279979 13884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:48.280369 13857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:48.280632 13857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:48.281780 13857 hybrid_clock.cc:648] HybridClock initialized: now 1790373948281750 us; error 45 us; skew 500 ppm
Sep 25 22:05:48 dist-test-slave-7mqr krb5kdc[13274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373948, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:48.284381 13857 init.cc:377] Logged in from keytab as oryx/127.5.0.65@KRBTEST.COM (short username oryx)
I20260925 22:05:48.284801 13857 webserver.cc:460] Webserver started at http://127.5.0.65:45833/ using document root <none> and password file <none>
I20260925 22:05:48.285055 13857 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:48.285141 13857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:48.286563 13857 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:05:48.287202 13891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:48.287391 13857 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:05:48.287493 13857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-0/wal
uuid: "82fca1cff13f4034bdb4ef3057d66fc4"
format_stamp: "Formatted at 2026-09-25 22:05:47 on dist-test-slave-7mqr"
server_key: "f8c7f35b73b1d35b38cad938ff591105"
server_key_iv: "8b958817c6a76e2d997a4df5f6826f40"
server_key_version: "encryptionkey@0"
I20260925 22:05:48.287835 13857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:48.303575 13857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:48.305794 13857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:48.305958 13857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:48.306193 13857 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:48.306502 13857 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:48.306555 13857 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:48.306600 13857 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:48.306627 13857 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:48.312723 13857 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.65:33749
I20260925 22:05:48.312767 14004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.65:33749 every 8 connection(s)
I20260925 22:05:48.313102 13857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
I20260925 22:05:48.314092 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 13857
I20260925 22:05:48.314397 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.66:42785
--local_ip_for_outbound_sockets=127.5.0.66
--tserver_master_addrs=127.5.0.126:38597
--webserver_port=33657
--webserver_interface=127.5.0.66
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:35089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.0.66
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 22:05:48 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:48 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:48.323493 14007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:48.313846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.65:43323)
0925 22:05:48.316003 (+ 2157us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:48.316040 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:48.316552 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:48.317011 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:48.317014 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:48.317684 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:48.317877 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:48.317882 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.319034 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.319035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:48.319830 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:48.319832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.320484 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.320485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:48.320616 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:48.321188 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:48.321196 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:48.323288 (+ 2092us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":2118,"thread_start_us":116,"threads_started":1}
W20260925 22:05:48.323542 14008 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:48.313911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:43323 (local address 127.5.0.126:38597)
0925 22:05:48.314114 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:48.314123 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:48.316162 (+ 2039us) server_negotiation.cc:407] Connection header received
0925 22:05:48.316781 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:48.316786 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:48.316837 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:48.316972 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:48.317994 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.318938 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.319990 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.320382 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.320511 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:48.323398 (+ 2887us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.65:43323: BlockingRecv error: recv got EOF from 127.5.0.65:43323 (error 108)
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":88,"threads_started":1}
W20260925 22:05:48.323750 14005 heartbeater.cc:644] Failed to heartbeat to 127.5.0.126:38597 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.0.126:38597: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Sep 25 22:05:48 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:48 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:48.328293 14010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:48.324051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.65:49513)
0925 22:05:48.324204 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:48.324214 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:48.324294 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:48.324455 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:48.324456 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:48.324593 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:48.324633 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:48.324635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.325240 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.325241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:48.325753 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:48.325754 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.326119 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.326120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:48.326178 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:48.326621 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:48.326625 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:48.328198 (+ 1573us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":98,"threads_started":1}
W20260925 22:05:48.328483 14008 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:48.324003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:49513 (local address 127.5.0.126:38597)
0925 22:05:48.324200 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:48.324202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:48.324267 (+ 65us) server_negotiation.cc:407] Connection header received
0925 22:05:48.324360 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:48.324361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:48.324372 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:48.324440 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:48.324713 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.325160 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.325832 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.326019 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.326087 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:48.328266 (+ 2179us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.65:49513: BlockingRecv error: recv got EOF from 127.5.0.65:49513 (error 108)
Metrics: {"server-negotiator.queue_time_us":176}
Sep 25 22:05:48 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:48 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:48.332753 14007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:48.328510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.65:46021)
0925 22:05:48.328709 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:48.328718 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:48.328786 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:48.328937 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:48.328937 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:48.329078 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:48.329116 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:48.329117 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.329707 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.329708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:48.330191 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:48.330192 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.330529 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.330530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:48.330597 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:48.330998 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:48.331008 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:48.332675 (+ 1667us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182}
W20260925 22:05:48.332790 14008 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:48.328592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:46021 (local address 127.5.0.126:38597)
0925 22:05:48.328679 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:48.328681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:48.328759 (+ 78us) server_negotiation.cc:407] Connection header received
0925 22:05:48.328852 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:48.328853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:48.328863 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:48.328935 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:48.329203 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.329629 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.330268 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.330449 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.330529 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:48.332730 (+ 2201us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.65:46021: BlockingRecv error: recv got EOF from 127.5.0.65:46021 (error 108)
Metrics: {"server-negotiator.queue_time_us":60,"thread_start_us":84,"threads_started":1}
W20260925 22:05:48.332970 14005 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 22:05:48.400008 14009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:48.400292 14009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:48.400352 14009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:48.401602 14009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:48.401643 14009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:48.401691 14009 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.66
I20260925 22:05:48.403139 14009 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:35089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.66.keytab
--principal=oryx/127.5.0.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.66:42785
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.0.66
--webserver_port=33657
--tserver_master_addrs=127.5.0.126:38597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14009
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.66
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:48.403378 14009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:48.403614 14009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:48.408463 14016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:48.408473 14017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:48.408488 14019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:48.408794 14009 server_base.cc:1034] running on GCE node
I20260925 22:05:48.408995 14009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:48.409245 14009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:48.410558 14009 hybrid_clock.cc:648] HybridClock initialized: now 1790373948410490 us; error 37 us; skew 500 ppm
Sep 25 22:05:48 dist-test-slave-7mqr krb5kdc[13274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373948, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:48.413235 14009 init.cc:377] Logged in from keytab as oryx/127.5.0.66@KRBTEST.COM (short username oryx)
I20260925 22:05:48.413650 14009 webserver.cc:460] Webserver started at http://127.5.0.66:33657/ using document root <none> and password file <none>
I20260925 22:05:48.413875 14009 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:48.413932 14009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:48.415369 14009 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:05:48.416234 14026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:48.416476 14009 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:05:48.416605 14009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-1/wal
uuid: "ecd0620b63eb473b9f4b9bbc689d5eba"
format_stamp: "Formatted at 2026-09-25 22:05:47 on dist-test-slave-7mqr"
server_key: "9e72c4f8ba9adfee757ab85425958e8d"
server_key_iv: "bc7c4dddf24fe353db8544dcc5c5dbd2"
server_key_version: "encryptionkey@0"
I20260925 22:05:48.417003 14009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:48.431452 14009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:48.434077 14009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:48.434252 14009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:48.434500 14009 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:48.434828 14009 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:48.434885 14009 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:48.434926 14009 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:48.434974 14009 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:48.441309 14009 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.66:42785
I20260925 22:05:48.441349 14139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.66:42785 every 8 connection(s)
I20260925 22:05:48.441725 14009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
I20260925 22:05:48.443133 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 14009
I20260925 22:05:48.443523 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.67:38355
--local_ip_for_outbound_sockets=127.5.0.67
--tserver_master_addrs=127.5.0.126:38597
--webserver_port=43279
--webserver_interface=127.5.0.67
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:35089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.0.67
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 22:05:48 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:48 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
W20260925 22:05:48.450388 14008 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:48.442500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:44681 (local address 127.5.0.126:38597)
0925 22:05:48.442562 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:48.442564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:48.442749 (+ 185us) server_negotiation.cc:407] Connection header received
0925 22:05:48.443289 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:48.443291 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:48.443310 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:48.443399 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:48.444488 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.445206 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.446325 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.446585 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.446662 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:48.450339 (+ 3677us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.66:44681: BlockingRecv error: recv got EOF from 127.5.0.66:44681 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 22:05:48.450496 14142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:48.442453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.66:44681)
0925 22:05:48.442614 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:48.442656 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:48.443151 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:48.443459 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:48.443463 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:48.444193 (+ 730us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:48.444386 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:48.444390 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.445312 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.445314 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:48.446206 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:48.446209 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.446686 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.446687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:48.446860 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:48.447441 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:48.447453 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:48.450241 (+ 2788us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":97,"threads_started":1}
W20260925 22:05:48.450691 14140 heartbeater.cc:644] Failed to heartbeat to 127.5.0.126:38597 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.0.126:38597: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Sep 25 22:05:48 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:48 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:48.457326 14142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:48.450930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.66:41085)
0925 22:05:48.450986 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:48.450998 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:48.451061 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:48.452439 (+ 1378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:48.452440 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:48.452570 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:48.452618 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:48.452619 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.453665 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.453666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:48.454265 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:48.454267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.454757 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.454758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:48.454813 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:48.455347 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:48.455352 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:48.457255 (+ 1903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 22:05:48.457427 14008 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:48.451060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:41085 (local address 127.5.0.126:38597)
0925 22:05:48.452306 (+ 1246us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:48.452308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:48.452318 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:05:48.452342 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:48.452344 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:48.452355 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:48.452424 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:48.452700 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.453493 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.454346 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.454651 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.454743 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:48.457337 (+ 2594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.66:41085: BlockingRecv error: recv got EOF from 127.5.0.66:41085 (error 108)
Metrics: {"server-negotiator.queue_time_us":1217}
Sep 25 22:05:48 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:48 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:48.462152 14142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:48.457545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.66:47413)
0925 22:05:48.457713 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:48.457723 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:48.457758 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:48.457877 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:48.457878 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:48.458018 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:48.458049 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:48.458050 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.458699 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.458700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:48.459505 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:48.459507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.459828 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.459829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:48.459879 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:48.460323 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:48.460328 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:48.462081 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":74,"threads_started":1}
W20260925 22:05:48.462273 14008 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:48.457702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:47413 (local address 127.5.0.126:38597)
0925 22:05:48.457751 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:48.457753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:48.457768 (+ 15us) server_negotiation.cc:407] Connection header received
0925 22:05:48.457808 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:48.457809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:48.457816 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:48.457861 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:48.458133 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.458624 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.459580 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.459764 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.459815 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:48.462182 (+ 2367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.66:47413: BlockingRecv error: recv got EOF from 127.5.0.66:47413 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260925 22:05:48.462334 14140 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 22:05:48.529488 14143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:05:48.529747 14143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:05:48.529790 14143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:05:48.531078 14143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:05:48.531149 14143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:05:48.531201 14143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.67
I20260925 22:05:48.532821 14143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:35089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/krb5kdc/oryx_127.5.0.67.keytab
--principal=oryx/127.5.0.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.67:38355
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.0.67
--webserver_port=43279
--tserver_master_addrs=127.5.0.126:38597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.67
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:05:48.533139 14143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:05:48.533393 14143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:05:48.538615 14150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:48.538622 14152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:05:48.538744 14149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:05:48.538774 14143 server_base.cc:1034] running on GCE node
I20260925 22:05:48.539076 14143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:05:48.539335 14143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:05:48.540488 14143 hybrid_clock.cc:648] HybridClock initialized: now 1790373948540469 us; error 37 us; skew 500 ppm
Sep 25 22:05:48 dist-test-slave-7mqr krb5kdc[13274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373948, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.0.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:05:48.542932 14143 init.cc:377] Logged in from keytab as oryx/127.5.0.67@KRBTEST.COM (short username oryx)
I20260925 22:05:48.543352 14143 webserver.cc:460] Webserver started at http://127.5.0.67:43279/ using document root <none> and password file <none>
I20260925 22:05:48.543573 14143 fs_manager.cc:359] Metadata directory not provided
I20260925 22:05:48.543660 14143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:05:48.545033 14143 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:05:48.545723 14159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:48.545917 14143 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:05:48.546013 14143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-2/wal
uuid: "8213f62c97ae4af99d96e67972042bf5"
format_stamp: "Formatted at 2026-09-25 22:05:47 on dist-test-slave-7mqr"
server_key: "e47a39a8f59ad259cf4b17bc472293b7"
server_key_iv: "c8a31e86bd3288fee5b1069ef0194ec8"
server_key_version: "encryptionkey@0"
I20260925 22:05:48.546360 14143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:05:48.560611 14143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:05:48.562522 14143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:05:48.562692 14143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:05:48.562942 14143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:05:48.563244 14143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:05:48.563297 14143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:48.563338 14143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:05:48.563366 14143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:05:48.569056 14143 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.67:38355
I20260925 22:05:48.569161 14272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.67:38355 every 8 connection(s)
I20260925 22:05:48.569463 14143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
Sep 25 22:05:48 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:48 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
W20260925 22:05:48.576622 14008 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:48.570217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:52297 (local address 127.5.0.126:38597)
0925 22:05:48.570300 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:48.570302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:48.570499 (+ 197us) server_negotiation.cc:407] Connection header received
0925 22:05:48.570994 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:48.570995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:48.571004 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:48.571046 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:48.571989 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.572457 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.573426 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.573598 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.573646 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:48.576564 (+ 2918us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.67:52297: BlockingRecv error: recv got EOF from 127.5.0.67:52297 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 22:05:48.576696 14275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:48.570184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.67:52297)
0925 22:05:48.570357 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:48.570392 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:48.570912 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:48.571115 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:48.571118 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:48.571671 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:48.571855 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:48.571859 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.572550 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.572551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:48.573353 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:48.573355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.573679 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.573680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:48.573786 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:48.574329 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:48.574341 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:48.576493 (+ 2152us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":94,"threads_started":1}
W20260925 22:05:48.576907 14273 heartbeater.cc:644] Failed to heartbeat to 127.5.0.126:38597 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.0.126:38597: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
I20260925 22:05:48.578833 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 14143
Sep 25 22:05:48 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:48 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:48.581416 14275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:48.577108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.67:53169)
0925 22:05:48.577157 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:48.577174 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:48.577230 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:48.577417 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:48.577418 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:48.577521 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:48.577563 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:48.577564 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.578148 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.578149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:48.578633 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:48.578634 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.579121 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.579122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:48.579204 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:48.579566 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:48.579570 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:48.581355 (+ 1785us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260925 22:05:48.581511 14008 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:48.577264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:53169 (local address 127.5.0.126:38597)
0925 22:05:48.577316 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:48.577318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:48.577325 (+ 7us) server_negotiation.cc:407] Connection header received
0925 22:05:48.577344 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:48.577344 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:48.577353 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:48.577404 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:48.577620 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.578087 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.578724 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.579014 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.579118 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:48.581447 (+ 2329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.67:53169: BlockingRecv error: recv got EOF from 127.5.0.67:53169 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Sep 25 22:05:48 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:48 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:48.585882 14275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:48.581673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.67:55853)
0925 22:05:48.581712 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:48.581721 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:48.581751 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:48.581858 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:48.581858 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:48.581954 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:48.581978 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:48.581978 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.582676 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.582677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:48.583153 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:48.583154 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.583606 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.583607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:48.583648 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:48.584064 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:48.584070 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:48.585806 (+ 1736us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260925 22:05:48.585922 14008 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:48.581727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:55853 (local address 127.5.0.126:38597)
0925 22:05:48.581760 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:48.581762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:48.581770 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:05:48.581790 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:48.581791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:48.581799 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:48.581856 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:48.582038 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.582599 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.583227 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.583529 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.583609 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:48.585864 (+ 2255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.67:55853: BlockingRecv error: recv got EOF from 127.5.0.67:55853 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260925 22:05:48.585996 14273 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 22:05:48.587323 14011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:48.579246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37886 (local address 127.5.0.126:38597)
0925 22:05:48.579313 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:48.579315 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:48.579692 (+ 377us) server_negotiation.cc:407] Connection header received
0925 22:05:48.579777 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:48.579778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:48.579789 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:48.579855 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:48.580648 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.581245 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.582369 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:48.582553 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:48.582613 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:48.584084 (+ 1471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:05:48.584089 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:05:48.584089 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:05:48.584102 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:05:48.585858 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:48.586320 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:48.586324 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:48.586324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:48.586426 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:05:48.586631 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:05:48.586632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:05:48.586632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:05:48.586865 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:05:48.587022 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:48.587120 (+ 98us) server_negotiation.cc:299] Negotiation successful
0925 22:05:48.587231 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
Sep 25 22:05:49 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:49 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:49.337805 14277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:49.333307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.65:46087)
0925 22:05:49.333448 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:49.333459 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:49.333511 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:49.333634 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:49.333635 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:49.333806 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:49.333853 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:49.333854 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:49.334581 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:49.334582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:49.335137 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:49.335139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:49.335501 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:49.335502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:49.335554 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:49.336072 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:49.336076 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:49.337739 (+ 1663us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":85,"threads_started":1}
W20260925 22:05:49.337947 14278 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:49.333398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:46087 (local address 127.5.0.126:38597)
0925 22:05:49.333500 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:49.333503 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:49.333512 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:05:49.333556 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:49.333557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:49.333565 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:49.333620 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:49.333921 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:49.334461 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:49.335222 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:49.335413 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:49.335474 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:49.337852 (+ 2378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.65:46087: BlockingRecv error: recv got EOF from 127.5.0.65:46087 (error 108)
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":56,"threads_started":1}
Sep 25 22:05:49 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:49 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:49.467386 14279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:49.462721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.66:39479)
0925 22:05:49.462846 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:49.462859 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:49.462928 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:49.463056 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:49.463057 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:49.463234 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:49.463279 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:49.463281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:49.463967 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:49.463968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:49.464561 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:49.464564 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:49.464965 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:49.464966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:49.465039 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:49.465465 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:49.465476 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:49.467278 (+ 1802us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
W20260925 22:05:49.467429 14278 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:49.462800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:39479 (local address 127.5.0.126:38597)
0925 22:05:49.462864 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:49.462866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:49.462911 (+ 45us) server_negotiation.cc:407] Connection header received
0925 22:05:49.462980 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:49.462981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:49.462992 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:49.463049 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:49.463359 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:49.463843 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:49.464652 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:49.464888 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:49.464968 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:49.467363 (+ 2395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.66:39479: BlockingRecv error: recv got EOF from 127.5.0.66:39479 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 25 22:05:49 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:49 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:49.591769 14280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:49.586344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.67:59313)
0925 22:05:49.586494 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:49.586508 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:49.586588 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:49.586803 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:49.586804 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:49.587025 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:49.587086 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:49.587088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:49.588002 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:49.588003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:49.588643 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:49.588645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:49.589032 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:49.589034 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:49.589089 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:49.589616 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:49.589621 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:49.591664 (+ 2043us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":53,"threads_started":1}
W20260925 22:05:49.591938 14278 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:49.586514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:59313 (local address 127.5.0.126:38597)
0925 22:05:49.586556 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:49.586559 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:49.586569 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:05:49.586689 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:49.586690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:49.586705 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:49.586758 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:49.587154 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:49.587936 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:49.588745 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:49.588951 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:49.589023 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:49.591782 (+ 2759us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.67:59313: BlockingRecv error: recv got EOF from 127.5.0.67:59313 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Sep 25 22:05:50 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:50 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:50.342705 14281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:50.338234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.65:34429)
0925 22:05:50.338379 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:50.338390 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:50.338437 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:50.338565 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:50.338565 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:50.338718 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:50.338767 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:50.338769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:50.339402 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:50.339403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:50.340006 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:50.340008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:50.340323 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:50.340324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:50.340361 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:50.340773 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:50.340777 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:50.342647 (+ 1870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":84,"threads_started":1}
W20260925 22:05:50.342844 14282 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:50.338300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:34429 (local address 127.5.0.126:38597)
0925 22:05:50.338425 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:50.338427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:50.338435 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:05:50.338490 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:50.338490 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:50.338500 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:50.338561 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:50.338822 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:50.339325 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:50.340070 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:50.340260 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:50.340310 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:50.342734 (+ 2424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.65:34429: BlockingRecv error: recv got EOF from 127.5.0.65:34429 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":75,"threads_started":1}
Sep 25 22:05:50 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:50 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:50.472440 14283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:50.467947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.66:35757)
0925 22:05:50.468096 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:50.468110 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:50.468171 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:50.468366 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:50.468367 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:50.468542 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:50.468587 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:50.468589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:50.469256 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:50.469257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:50.469789 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:50.469791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:50.470154 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:50.470154 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:50.470211 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:50.470676 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:50.470680 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:50.472346 (+ 1666us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
W20260925 22:05:50.472486 14282 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:50.468178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:35757 (local address 127.5.0.126:38597)
0925 22:05:50.468242 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:50.468244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:50.468251 (+ 7us) server_negotiation.cc:407] Connection header received
0925 22:05:50.468275 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:50.468276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:50.468289 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:50.468358 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:50.468669 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:50.469174 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:50.469884 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:50.470072 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:50.470154 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:50.472423 (+ 2269us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.66:35757: BlockingRecv error: recv got EOF from 127.5.0.66:35757 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 25 22:05:50 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:50 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:50.596872 14284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:50.592219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.67:57003)
0925 22:05:50.592361 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:50.592375 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:50.592445 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:50.592570 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:50.592571 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:50.592733 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:50.592782 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:50.592784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:50.593688 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:50.593689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:50.594203 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:50.594204 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:50.594507 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:50.594508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:50.594550 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:50.595002 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:50.595006 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:50.596776 (+ 1770us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":67,"threads_started":1}
W20260925 22:05:50.596908 14282 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:50.592348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:57003 (local address 127.5.0.126:38597)
0925 22:05:50.592408 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:50.592410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:50.592429 (+ 19us) server_negotiation.cc:407] Connection header received
0925 22:05:50.592497 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:50.592498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:50.592508 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:50.592548 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:50.592846 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:50.593606 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:50.594269 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:50.594441 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:50.594488 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:50.596849 (+ 2361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.67:57003: BlockingRecv error: recv got EOF from 127.5.0.67:57003 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 25 22:05:51 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:51 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:51.348261 14285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:51.343203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.65:57795)
0925 22:05:51.343466 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:51.343479 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:51.343559 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:51.343704 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:51.343704 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:51.343856 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:51.343953 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:51.343955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:51.344897 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:51.344898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:51.345425 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:51.345426 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:51.345769 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:51.345770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:51.345807 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:51.346309 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:51.346314 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:51.348205 (+ 1891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":58,"threads_started":1}
W20260925 22:05:51.348423 14286 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:51.343423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:57795 (local address 127.5.0.126:38597)
0925 22:05:51.343572 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:51.343574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:51.343583 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:05:51.343607 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:51.343607 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:51.343617 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:51.343679 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:51.344076 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:51.344829 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:51.345501 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:51.345702 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:51.345759 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:51.348327 (+ 2568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.65:57795: BlockingRecv error: recv got EOF from 127.5.0.65:57795 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":70,"threads_started":1}
Sep 25 22:05:51 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:51 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:51.477820 14287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:51.472951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.66:35881)
0925 22:05:51.473088 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:51.473099 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:51.473165 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:51.473294 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:51.473294 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:51.473453 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:51.473501 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:51.473503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:51.474142 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:51.474143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:51.474701 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:51.474703 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:51.475029 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:51.475029 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:51.475066 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:51.475594 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:51.475599 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:51.477751 (+ 2152us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":71,"threads_started":1}
W20260925 22:05:51.477937 14286 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:51.473037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:35881 (local address 127.5.0.126:38597)
0925 22:05:51.473108 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:51.473110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:51.473153 (+ 43us) server_negotiation.cc:407] Connection header received
0925 22:05:51.473209 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:51.473210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:51.473220 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:51.473272 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:51.473572 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:51.474070 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:51.474784 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:51.474964 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:51.475033 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:51.477843 (+ 2810us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.66:35881: BlockingRecv error: recv got EOF from 127.5.0.66:35881 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 25 22:05:51 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:51 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:51.602068 14288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:51.597360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.67:49025)
0925 22:05:51.597505 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:51.597516 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:51.597579 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:51.597704 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:51.597705 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:51.597863 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:51.597912 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:51.597913 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:51.598513 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:51.598513 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:51.599035 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:51.599036 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:51.599330 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:51.599330 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:51.599366 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:51.599865 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:51.599870 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:51.601959 (+ 2089us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":100,"threads_started":1}
W20260925 22:05:51.602095 14286 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:51.597415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:49025 (local address 127.5.0.126:38597)
0925 22:05:51.597478 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:51.597480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:51.597580 (+ 100us) server_negotiation.cc:407] Connection header received
0925 22:05:51.597621 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:51.597621 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:51.597634 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:51.597685 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:51.597972 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:51.598449 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:51.599100 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:51.599269 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:51.599320 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:51.602033 (+ 2713us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.67:49025: BlockingRecv error: recv got EOF from 127.5.0.67:49025 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 25 22:05:52 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:52 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:52.352972 14289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:52.348665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.65:42523)
0925 22:05:52.348809 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:52.348832 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:52.348894 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:52.349059 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:52.349060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:52.349214 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:52.349261 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:52.349263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:52.349893 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:52.349894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:52.350446 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:52.350448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:52.350773 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:52.350774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:52.350812 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:52.351240 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:52.351244 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:52.352913 (+ 1669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":88,"threads_started":1}
W20260925 22:05:52.353114 14290 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:52.348771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:42523 (local address 127.5.0.126:38597)
0925 22:05:52.348870 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:52.348872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:52.348888 (+ 16us) server_negotiation.cc:407] Connection header received
0925 22:05:52.348946 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:52.348946 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:52.348955 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:52.349053 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:52.349317 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:52.349833 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:52.350514 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:52.350708 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:52.350764 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:52.353005 (+ 2241us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.65:42523: BlockingRecv error: recv got EOF from 127.5.0.65:42523 (error 108)
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":53,"threads_started":1}
Sep 25 22:05:52 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:52 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:52.486692 14291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:52.478304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.66:40733)
0925 22:05:52.478460 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:52.478471 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:52.478561 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:52.478720 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:52.478720 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:52.478902 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:52.478946 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:52.478948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:52.479621 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:52.479622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:52.480314 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:52.480317 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:52.480662 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:52.480663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:52.480728 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:52.484898 (+ 4170us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:52.484903 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:52.486635 (+ 1732us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":82,"threads_started":1}
W20260925 22:05:52.486799 14290 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:52.478370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:40733 (local address 127.5.0.126:38597)
0925 22:05:52.478422 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:52.478424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:52.478546 (+ 122us) server_negotiation.cc:407] Connection header received
0925 22:05:52.478618 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:52.478619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:52.478633 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:52.478715 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:52.479030 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:52.479535 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:52.480397 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:52.480587 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:52.480665 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:52.486729 (+ 6064us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.66:40733: BlockingRecv error: recv got EOF from 127.5.0.66:40733 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Sep 25 22:05:52 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:52 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:52.606963 14292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:52.602509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.67:56105)
0925 22:05:52.602654 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:52.602671 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:52.602727 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:52.602860 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:52.602861 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:52.603036 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:52.603083 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:52.603084 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:52.603736 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:52.603737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:52.604281 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:52.604282 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:52.604635 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:52.604636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:52.604674 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:52.605182 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:52.605186 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:52.606907 (+ 1721us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":78,"threads_started":1}
W20260925 22:05:52.607097 14290 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:52.602604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:56105 (local address 127.5.0.126:38597)
0925 22:05:52.602684 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:52.602686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:52.602722 (+ 36us) server_negotiation.cc:407] Connection header received
0925 22:05:52.602779 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:52.602779 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:52.602802 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:52.602842 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:52.603173 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:52.603672 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:52.604376 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:52.604566 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:52.604630 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:52.607003 (+ 2373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.67:56105: BlockingRecv error: recv got EOF from 127.5.0.67:56105 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 25 22:05:53 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:53 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:53.358143 14293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:53.353428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.65:54889)
0925 22:05:53.353556 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:53.353567 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:53.353628 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:53.353811 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:53.353812 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:53.354001 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:53.354048 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:53.354050 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:53.354821 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:53.354822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:53.355383 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:53.355385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:53.355740 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:53.355741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:53.355782 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:53.356253 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:53.356258 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:53.358086 (+ 1828us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":77,"threads_started":1}
W20260925 22:05:53.358263 14294 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:53.353498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:54889 (local address 127.5.0.126:38597)
0925 22:05:53.353623 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:53.353625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:53.353634 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:05:53.353716 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:53.353716 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:53.353727 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:53.353796 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:53.354141 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:53.354748 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:53.355466 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:53.355666 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:53.355726 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:53.358181 (+ 2455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.65:54889: BlockingRecv error: recv got EOF from 127.5.0.65:54889 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":71,"threads_started":1}
Sep 25 22:05:53 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:53 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:53.491834 14295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:53.487213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.66:49901)
0925 22:05:53.487327 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:53.487339 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:53.487398 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:53.487547 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:53.487548 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:53.487719 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:53.487764 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:53.487766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:53.488476 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:53.488477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:53.489000 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:53.489001 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:53.489369 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:53.489370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:53.489434 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:53.489917 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:53.489922 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:53.491774 (+ 1852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":61,"threads_started":1}
W20260925 22:05:53.491984 14294 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:53.487376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:49901 (local address 127.5.0.126:38597)
0925 22:05:53.487418 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:53.487420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:53.487429 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:05:53.487457 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:53.487457 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:53.487476 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:53.487542 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:53.487844 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:53.488385 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:53.489080 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:53.489283 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:53.489368 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:53.491883 (+ 2515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.66:49901: BlockingRecv error: recv got EOF from 127.5.0.66:49901 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Sep 25 22:05:53 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:53 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:53.612411 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:53.607419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.67:56047)
0925 22:05:53.607567 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:53.607578 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:53.607671 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:53.607830 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:53.607831 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:53.608033 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:53.608078 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:53.608080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:53.608738 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:53.608739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:53.609278 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:53.609279 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:53.609832 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:53.609833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:53.609895 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:53.610447 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:53.610452 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:53.612353 (+ 1901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":72,"threads_started":1}
W20260925 22:05:53.612627 14294 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:53.607455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:56047 (local address 127.5.0.126:38597)
0925 22:05:53.607517 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:53.607519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:53.607652 (+ 133us) server_negotiation.cc:407] Connection header received
0925 22:05:53.607730 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:53.607731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:53.607750 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:53.607824 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:53.608157 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:53.608640 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:53.609376 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:53.609708 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:53.609832 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:53.612497 (+ 2665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.67:56047: BlockingRecv error: recv got EOF from 127.5.0.67:56047 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Sep 25 22:05:54 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:54 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:54.363135 14297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:54.358591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.65:59119)
0925 22:05:54.358719 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:54.358733 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:54.358787 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:54.358932 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:54.358933 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:54.359087 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:54.359135 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:54.359137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:54.359812 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:54.359812 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:54.360385 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:54.360387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:54.360754 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:54.360755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:54.360795 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:54.361305 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:54.361309 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:54.363079 (+ 1770us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":89,"threads_started":1}
W20260925 22:05:54.363291 14298 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:54.358680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:59119 (local address 127.5.0.126:38597)
0925 22:05:54.358798 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:54.358800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:54.358809 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:05:54.358834 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:54.358834 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:54.358853 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:54.358919 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:54.359199 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:54.359747 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:54.360466 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:54.360685 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:54.360745 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:54.363189 (+ 2444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.65:59119: BlockingRecv error: recv got EOF from 127.5.0.65:59119 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":68,"threads_started":1}
Sep 25 22:05:54 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:54 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:54.496915 14299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:54.492346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.66:33101)
0925 22:05:54.492500 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:54.492511 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:54.492574 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:54.492733 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:54.492733 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:54.492889 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:54.492938 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:54.492961 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:54.493695 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:54.493696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:54.494195 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:54.494197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:54.494546 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:54.494547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:54.494583 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:54.495047 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:54.495051 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:54.496821 (+ 1770us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":96,"threads_started":1}
W20260925 22:05:54.497151 14298 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:54.492400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:33101 (local address 127.5.0.126:38597)
0925 22:05:54.492470 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:54.492472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:54.492576 (+ 104us) server_negotiation.cc:407] Connection header received
0925 22:05:54.492623 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:54.492623 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:54.492650 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:54.492706 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:54.493075 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:54.493613 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:54.494269 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:54.494450 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:54.494538 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:54.496961 (+ 2423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.66:33101: BlockingRecv error: recv got EOF from 127.5.0.66:33101 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 25 22:05:54 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:54 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:54.618345 14300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:54.612976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.67:46809)
0925 22:05:54.613165 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:54.613176 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:54.613275 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:54.613453 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:54.613454 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:54.613645 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:54.613694 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:54.613696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:54.614415 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:54.614417 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:54.615000 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:54.615002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:54.615412 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:54.615413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:54.615478 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:54.616112 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:54.616117 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:54.618283 (+ 2166us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":112,"threads_started":1}
W20260925 22:05:54.618472 14298 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:54.613055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:46809 (local address 127.5.0.126:38597)
0925 22:05:54.613129 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:54.613132 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:54.613254 (+ 122us) server_negotiation.cc:407] Connection header received
0925 22:05:54.613334 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:54.613336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:54.613357 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:54.613430 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:54.613785 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:54.614324 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:54.615106 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:54.615316 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:54.615412 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:54.618392 (+ 2980us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.67:46809: BlockingRecv error: recv got EOF from 127.5.0.67:46809 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 25 22:05:55 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:55 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:55.368372 14301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:55.363675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.65:36785)
0925 22:05:55.363819 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:55.363831 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:55.363905 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:55.364058 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:55.364059 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:55.364221 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:55.364267 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:55.364269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:55.364943 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:55.364945 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:55.365658 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:55.365661 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:55.366021 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:55.366023 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:55.366081 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:55.366584 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:55.366589 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:55.368279 (+ 1690us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":88,"threads_started":1}
W20260925 22:05:55.368438 14302 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:55.363735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:36785 (local address 127.5.0.126:38597)
0925 22:05:55.363864 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:55.363866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:55.363876 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:05:55.363956 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:55.363957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:55.363967 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:55.364031 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:55.364333 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:55.364825 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:55.365752 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:55.365948 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:55.366021 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:55.368366 (+ 2345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.65:36785: BlockingRecv error: recv got EOF from 127.5.0.65:36785 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":73,"threads_started":1}
Sep 25 22:05:55 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:55 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:55.501766 14303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:55.497360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.66:56545)
0925 22:05:55.497471 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:55.497483 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:55.497557 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:55.497689 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:55.497690 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:55.497848 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:55.497897 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:55.497898 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:55.498523 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:55.498524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:55.499048 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:55.499049 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:55.499357 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:55.499358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:55.499397 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:55.499839 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:55.499844 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:55.501708 (+ 1864us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":53,"threads_started":1}
W20260925 22:05:55.501860 14302 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:55.497434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:56545 (local address 127.5.0.126:38597)
0925 22:05:55.497509 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:55.497511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:55.497549 (+ 38us) server_negotiation.cc:407] Connection header received
0925 22:05:55.497612 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:55.497612 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:55.497623 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:55.497670 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:55.497954 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:55.498447 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:55.499112 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:55.499294 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:55.499342 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:55.501794 (+ 2452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.66:56545: BlockingRecv error: recv got EOF from 127.5.0.66:56545 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 25 22:05:55 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:55 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:55.623755 14304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:55.618876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.67:53897)
0925 22:05:55.619066 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:55.619078 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:55.619171 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:55.619332 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:55.619333 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:55.619552 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:55.619600 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:55.619602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:55.620346 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:55.620348 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:55.620973 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:55.620975 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:55.621338 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:55.621339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:55.621397 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:55.621856 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:55.621861 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:55.623658 (+ 1797us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":104,"threads_started":1}
W20260925 22:05:55.623801 14302 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:55.618993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:53897 (local address 127.5.0.126:38597)
0925 22:05:55.619066 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:55.619069 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:55.619150 (+ 81us) server_negotiation.cc:407] Connection header received
0925 22:05:55.619237 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:55.619238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:55.619253 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:55.619326 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:55.619683 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:55.620254 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:55.621069 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:55.621256 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:55.621338 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:55.623737 (+ 2399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.67:53897: BlockingRecv error: recv got EOF from 127.5.0.67:53897 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 25 22:05:56 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:56 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:56.373342 14305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:56.368868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.65:46863)
0925 22:05:56.369009 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:56.369025 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:56.369076 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:56.369246 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:56.369247 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:56.369404 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:56.369451 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:56.369453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:56.370077 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:56.370078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:56.370594 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:56.370595 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:56.370985 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:56.370986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:56.371033 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:56.371474 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:56.371478 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:56.373271 (+ 1793us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":58,"threads_started":1}
W20260925 22:05:56.373471 14306 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:56.369021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:46863 (local address 127.5.0.126:38597)
0925 22:05:56.369134 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:56.369136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:56.369143 (+ 7us) server_negotiation.cc:407] Connection header received
0925 22:05:56.369163 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:56.369163 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:56.369172 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:56.369229 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:56.369513 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:56.370016 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:56.370670 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:56.370848 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:56.370896 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:56.373363 (+ 2467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.65:46863: BlockingRecv error: recv got EOF from 127.5.0.65:46863 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":60,"threads_started":1}
Sep 25 22:05:56 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:56 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:56.506971 14307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:56.502229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.66:50703)
0925 22:05:56.502355 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:56.502368 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:56.502442 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:56.502558 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:56.502559 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:56.502723 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:56.502773 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:56.502774 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:56.503532 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:56.503533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:56.504126 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:56.504129 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:56.504449 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:56.504450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:56.504490 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:56.504926 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:56.504930 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:56.506909 (+ 1979us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":66,"threads_started":1}
W20260925 22:05:56.507045 14306 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:56.502304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:50703 (local address 127.5.0.126:38597)
0925 22:05:56.502383 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:56.502384 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:56.502425 (+ 41us) server_negotiation.cc:407] Connection header received
0925 22:05:56.502489 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:56.502490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:56.502501 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:56.502548 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:56.502834 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:56.503462 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:56.504207 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:56.504385 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:56.504437 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:56.506999 (+ 2562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.66:50703: BlockingRecv error: recv got EOF from 127.5.0.66:50703 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 25 22:05:56 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:56 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:56.628855 14308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:56.624230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.67:44489)
0925 22:05:56.624394 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:56.624410 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:56.624489 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:56.624634 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:56.624635 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:56.624816 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:56.624858 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:56.624860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:56.625513 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:56.625514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:56.626037 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:56.626039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:56.626400 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:56.626401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:56.626472 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:56.626926 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:56.626931 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:56.628758 (+ 1827us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":110,"threads_started":1}
W20260925 22:05:56.628894 14306 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:56.624297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:44489 (local address 127.5.0.126:38597)
0925 22:05:56.624377 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:56.624379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:56.624479 (+ 100us) server_negotiation.cc:407] Connection header received
0925 22:05:56.624544 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:56.624545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:56.624560 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:56.624631 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:56.624940 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:56.625431 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:56.626122 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:56.626313 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:56.626397 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:56.628834 (+ 2437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.67:44489: BlockingRecv error: recv got EOF from 127.5.0.67:44489 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 25 22:05:57 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:57 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:57.378290 14309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:57.373781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.65:49233)
0925 22:05:57.373908 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:57.373919 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:57.373968 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:57.374094 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:57.374095 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:57.374260 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:57.374308 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:57.374310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:57.374932 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:57.374933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:57.375455 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:57.375457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:57.375772 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:57.375772 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:57.375813 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:57.376476 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:57.376481 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:57.378231 (+ 1750us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":81,"threads_started":1}
W20260925 22:05:57.378420 14310 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:57.373866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:49233 (local address 127.5.0.126:38597)
0925 22:05:57.373993 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:57.373995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:57.374002 (+ 7us) server_negotiation.cc:407] Connection header received
0925 22:05:57.374022 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:57.374022 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:57.374032 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:57.374083 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:57.374369 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:57.374866 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:57.375522 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:57.375709 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:57.375758 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:57.378333 (+ 2575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.65:49233: BlockingRecv error: recv got EOF from 127.5.0.65:49233 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":71,"threads_started":1}
Sep 25 22:05:57 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:57 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:57.516628 14311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:57.507598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.66:37967)
0925 22:05:57.507792 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:57.507803 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:57.507891 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:57.508097 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:57.508098 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:57.508275 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:57.508322 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:57.508324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:57.508992 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:57.508993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:57.509552 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:57.509554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:57.509947 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:57.509947 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:57.510004 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:57.514551 (+ 4547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:57.514557 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:57.516508 (+ 1951us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":118,"threads_started":1}
W20260925 22:05:57.516656 14310 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:57.507699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:37967 (local address 127.5.0.126:38597)
0925 22:05:57.507785 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:57.507788 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:57.507874 (+ 86us) server_negotiation.cc:407] Connection header received
0925 22:05:57.507981 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:57.507982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:57.507995 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:57.508090 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:57.508398 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:57.508905 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:57.509653 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:57.509861 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:57.509945 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:57.516598 (+ 6653us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.66:37967: BlockingRecv error: recv got EOF from 127.5.0.66:37967 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 25 22:05:57 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:57 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:57.634325 14312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:57.629439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.67:41887)
0925 22:05:57.629641 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:57.629652 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:57.629740 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:57.629902 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:57.629904 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:57.630090 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:57.630142 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:57.630144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:57.630809 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:57.630810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:57.631343 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:57.631345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:57.631693 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:57.631694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:57.631751 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:57.632317 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:57.632322 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:57.634231 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":111,"threads_started":1}
W20260925 22:05:57.634359 14310 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:57.629546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:41887 (local address 127.5.0.126:38597)
0925 22:05:57.629629 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:57.629631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:57.629719 (+ 88us) server_negotiation.cc:407] Connection header received
0925 22:05:57.629800 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:57.629801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:57.629816 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:57.629895 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:57.630220 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:57.630723 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:57.631423 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:57.631618 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:57.631697 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:57.634300 (+ 2603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.67:41887: BlockingRecv error: recv got EOF from 127.5.0.67:41887 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 25 22:05:58 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:58 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:58.383458 14313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:58.378839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.65:39975)
0925 22:05:58.378983 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:58.378994 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:58.379065 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:58.379203 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:58.379204 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:58.379365 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:58.379414 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:58.379415 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:58.380151 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:58.380152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:58.380709 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:58.380711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:58.381030 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:58.381031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:58.381070 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:58.381568 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:58.381573 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:58.383396 (+ 1823us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
W20260925 22:05:58.383592 14314 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:58.378892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:39975 (local address 127.5.0.126:38597)
0925 22:05:58.379043 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:58.379045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:58.379053 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:05:58.379109 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:58.379110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:58.379120 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:58.379187 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:58.379476 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:58.380062 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:58.380778 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:58.380963 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:58.381012 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:58.383482 (+ 2470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.65:39975: BlockingRecv error: recv got EOF from 127.5.0.65:39975 (error 108)
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":91,"threads_started":1}
Sep 25 22:05:58 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:58 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:58.521654 14315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:58.517138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.66:34713)
0925 22:05:58.517293 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:58.517304 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:58.517370 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:58.517546 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:58.517546 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:58.517721 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:58.517766 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:58.517767 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:58.518377 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:58.518377 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:58.518901 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:58.518903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:58.519221 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:58.519221 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:58.519260 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:58.519784 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:58.519789 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:58.521596 (+ 1807us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":112,"threads_started":1}
W20260925 22:05:58.521734 14314 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:58.517211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:34713 (local address 127.5.0.126:38597)
0925 22:05:58.517280 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:58.517282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:58.517363 (+ 81us) server_negotiation.cc:407] Connection header received
0925 22:05:58.517423 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:58.517424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:58.517436 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:58.517532 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:58.517827 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:58.518314 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:58.518971 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:58.519157 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:58.519211 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:58.521688 (+ 2477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.66:34713: BlockingRecv error: recv got EOF from 127.5.0.66:34713 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 25 22:05:58 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:58 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:58.639528 14316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:58.634893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.67:55707)
0925 22:05:58.635058 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:58.635070 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:58.635134 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:58.635314 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:58.635315 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:58.635493 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:58.635540 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:58.635542 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:58.636259 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:58.636260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:58.636782 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:58.636784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:58.637155 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:58.637156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:58.637224 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:58.637715 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:58.637721 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:58.639469 (+ 1748us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":105,"threads_started":1}
W20260925 22:05:58.639622 14314 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:58.635030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:55707 (local address 127.5.0.126:38597)
0925 22:05:58.635105 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:58.635107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:58.635116 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:05:58.635224 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:58.635225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:58.635245 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:58.635305 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:58.635622 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:58.636180 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:58.636869 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:58.637056 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:58.637155 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:58.639558 (+ 2403us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.67:55707: BlockingRecv error: recv got EOF from 127.5.0.67:55707 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 25 22:05:59 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:59 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:59.388280 14317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:59.383939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.65:43187)
0925 22:05:59.384087 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:59.384099 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:59.384180 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:59.384352 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:59.384353 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:59.384510 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:59.384557 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:59.384558 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:59.385212 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:59.385213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:59.385714 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:59.385716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:59.386047 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:59.386048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:59.386084 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:59.386541 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:59.386545 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:59.388223 (+ 1678us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":54,"threads_started":1}
W20260925 22:05:59.388386 14318 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:59.384122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:43187 (local address 127.5.0.126:38597)
0925 22:05:59.384235 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:59.384237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:59.384244 (+ 7us) server_negotiation.cc:407] Connection header received
0925 22:05:59.384264 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:59.384264 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:59.384274 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:59.384331 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:59.384631 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:59.385145 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:59.385787 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:59.385978 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:59.386047 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:59.388314 (+ 2267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.65:43187: BlockingRecv error: recv got EOF from 127.5.0.65:43187 (error 108)
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":61,"threads_started":1}
Sep 25 22:05:59 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:59 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:59.526856 14319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:59.522260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.66:45155)
0925 22:05:59.522416 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:59.522428 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:59.522518 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:59.522690 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:59.522691 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:59.522867 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:59.522911 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:59.522912 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:59.523588 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:59.523589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:59.524149 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:59.524151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:59.524522 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:59.524523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:59.524580 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:59.525064 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:59.525068 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:59.526760 (+ 1692us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":85,"threads_started":1}
W20260925 22:05:59.526906 14318 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:59.522345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:45155 (local address 127.5.0.126:38597)
0925 22:05:59.522394 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:59.522397 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:59.522504 (+ 107us) server_negotiation.cc:407] Connection header received
0925 22:05:59.522576 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:59.522577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:59.522590 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:59.522656 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:59.522992 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:59.523501 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:59.524236 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:59.524435 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:59.524521 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:59.526841 (+ 2320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.66:45155: BlockingRecv error: recv got EOF from 127.5.0.66:45155 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Sep 25 22:05:59 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:05:59 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:05:59.646311 14320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:59.640063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.67:54103)
0925 22:05:59.640241 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0925 22:05:59.640253 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:05:59.640319 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:05:59.640489 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:05:59.640489 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:05:59.640713 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:05:59.640770 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:59.640772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:59.641812 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:59.641813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:59.642721 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:05:59.642723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:05:59.643227 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:05:59.643228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:05:59.643282 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:59.643932 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:05:59.643937 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:05:59.646222 (+ 2285us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":97,"threads_started":1}
W20260925 22:05:59.646421 14318 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:59.640149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:54103 (local address 127.5.0.126:38597)
0925 22:05:59.640217 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:59.640219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:59.640320 (+ 101us) server_negotiation.cc:407] Connection header received
0925 22:05:59.640377 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:59.640378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:59.640397 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:59.640475 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:59.640857 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:59.641736 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:59.642806 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:59.643148 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:59.643238 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:59.646346 (+ 3108us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.67:54103: BlockingRecv error: recv got EOF from 127.5.0.67:54103 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 25 22:06:00 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:06:00 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:06:00.393536 14321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:00.388899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.65:47293)
0925 22:06:00.389057 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:00.389068 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:06:00.389128 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:06:00.389294 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:06:00.389295 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:06:00.389458 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:06:00.389506 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:00.389508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:00.390196 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:00.390197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:00.390737 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:00.390738 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:00.391067 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:00.391067 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:00.391105 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:00.391671 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:06:00.391677 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:06:00.393472 (+ 1795us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":92,"threads_started":1}
W20260925 22:06:00.393646 14322 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:06:00.388977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:47293 (local address 127.5.0.126:38597)
0925 22:06:00.389102 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:00.389108 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:00.389118 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:06:00.389197 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:00.389197 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:00.389208 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:00.389271 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:06:00.389573 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:00.390127 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:00.390808 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:00.390993 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:00.391064 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:00.393571 (+ 2507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.65:47293: BlockingRecv error: recv got EOF from 127.5.0.65:47293 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":72,"threads_started":1}
Sep 25 22:06:00 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:06:00 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:06:00.532220 14323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:00.527337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.66:51219)
0925 22:06:00.527506 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:00.527519 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:06:00.527592 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:06:00.527762 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:06:00.527763 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:06:00.527971 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:06:00.528028 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:00.528030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:00.528707 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:00.528708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:00.529246 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:00.529247 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:00.529612 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:00.529613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:00.529671 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:00.530222 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:06:00.530227 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:06:00.532149 (+ 1922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":87,"threads_started":1}
W20260925 22:06:00.532316 14322 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:06:00.527442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:51219 (local address 127.5.0.126:38597)
0925 22:06:00.527545 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:00.527547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:00.527573 (+ 26us) server_negotiation.cc:407] Connection header received
0925 22:06:00.527666 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:00.527667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:00.527680 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:00.527756 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:06:00.528119 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:00.528625 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:00.529331 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:00.529538 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:00.529615 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:00.532245 (+ 2630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.66:51219: BlockingRecv error: recv got EOF from 127.5.0.66:51219 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 25 22:06:00 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:06:00 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:06:00.652076 14324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:00.647117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.67:41667)
0925 22:06:00.647285 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:00.647301 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 22:06:00.647384 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:06:00.647534 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:06:00.647535 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:06:00.647727 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:06:00.647773 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:00.647774 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:00.648490 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:00.648491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:00.649112 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:00.649113 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:00.649475 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:00.649476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:00.649532 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:00.649995 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:06:00.649999 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:06:00.652008 (+ 2009us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":92,"threads_started":1}
W20260925 22:06:00.652176 14322 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:06:00.647254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:41667 (local address 127.5.0.126:38597)
0925 22:06:00.647310 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:00.647312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:00.647364 (+ 52us) server_negotiation.cc:407] Connection header received
0925 22:06:00.647441 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:00.647442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:00.647457 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:00.647528 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:06:00.647861 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:00.648395 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:00.649194 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:00.649387 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:00.649472 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:00.652098 (+ 2626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.67:41667: BlockingRecv error: recv got EOF from 127.5.0.67:41667 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Sep 25 22:06:01 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:06:01 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:06:01.398769 14325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:01.394021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.65:58057)
0925 22:06:01.394179 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:01.394190 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:06:01.394247 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:06:01.394423 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:06:01.394424 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:06:01.394607 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:06:01.394654 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:01.394656 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:01.395408 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:01.395409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:01.396005 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:01.396008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:01.396318 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:01.396319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:01.396355 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:01.396833 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:06:01.396838 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:06:01.398686 (+ 1848us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":90,"threads_started":1}
W20260925 22:06:01.398900 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:06:01.394095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:58057 (local address 127.5.0.126:38597)
0925 22:06:01.394227 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:01.394228 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:01.394236 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:06:01.394337 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:01.394338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:01.394348 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:01.394398 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:06:01.394731 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:01.395249 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:01.396071 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:01.396249 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:01.396297 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:01.398790 (+ 2493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.65:58057: BlockingRecv error: recv got EOF from 127.5.0.65:58057 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
Sep 25 22:06:01 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:06:01 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:06:01.537424 14327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:01.532789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.66:55771)
0925 22:06:01.532951 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:01.532982 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 22:06:01.533051 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:06:01.533243 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:06:01.533245 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:06:01.533421 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:06:01.533466 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:01.533467 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:01.534130 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:01.534131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:01.534653 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:01.534655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:01.535022 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:01.535023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:01.535088 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:01.535534 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:06:01.535539 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:06:01.537349 (+ 1810us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":95,"threads_started":1}
W20260925 22:06:01.537546 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:06:01.532930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:55771 (local address 127.5.0.126:38597)
0925 22:06:01.533007 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:01.533009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:01.533034 (+ 25us) server_negotiation.cc:407] Connection header received
0925 22:06:01.533131 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:01.533132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:01.533145 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:01.533235 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:06:01.533546 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:01.534040 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:01.534745 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:01.534936 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:01.535022 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:01.537444 (+ 2422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.66:55771: BlockingRecv error: recv got EOF from 127.5.0.66:55771 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 25 22:06:01 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:06:01 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:06:01.657014 14328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:01.652578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.67:54607)
0925 22:06:01.652737 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:01.652748 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:06:01.652804 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:06:01.652919 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:06:01.652920 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:06:01.653075 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:06:01.653122 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:01.653124 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:01.653798 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:01.653799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:01.654368 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:01.654370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:01.654726 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:01.654726 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:01.654784 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:01.655292 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:06:01.655296 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:06:01.656947 (+ 1651us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":99,"threads_started":1}
W20260925 22:06:01.657146 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:06:01.652596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:54607 (local address 127.5.0.126:38597)
0925 22:06:01.652669 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:01.652672 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:01.652810 (+ 138us) server_negotiation.cc:407] Connection header received
0925 22:06:01.652840 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:01.652841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:01.652853 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:01.652908 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:06:01.653193 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:01.653730 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:01.654466 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:01.654657 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:01.654720 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:01.657040 (+ 2320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.67:54607: BlockingRecv error: recv got EOF from 127.5.0.67:54607 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 25 22:06:02 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:06:02 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:06:02.404127 14329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:02.399201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.65:38601)
0925 22:06:02.399334 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:02.399345 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:06:02.399397 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:06:02.399548 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:06:02.399549 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:06:02.399759 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:06:02.399807 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:02.399809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:02.400652 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:02.400676 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:02.401357 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:02.401359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:02.401753 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:02.401754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:02.401794 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:02.402247 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:06:02.402251 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:06:02.404058 (+ 1807us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":83,"threads_started":1}
W20260925 22:06:02.404253 14330 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:06:02.399303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:38601 (local address 127.5.0.126:38597)
0925 22:06:02.399430 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:02.399432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:02.399440 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:06:02.399461 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:02.399461 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:02.399471 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:02.399533 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:06:02.399982 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:02.400587 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:02.401426 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:02.401688 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:02.401744 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:02.404174 (+ 2430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.65:38601: BlockingRecv error: recv got EOF from 127.5.0.65:38601 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":71,"threads_started":1}
Sep 25 22:06:02 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:06:02 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:06:02.547693 14331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:02.538080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.66:51661)
0925 22:06:02.538258 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:02.538275 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 22:06:02.538336 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:06:02.538485 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:06:02.538486 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:06:02.538643 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:06:02.538701 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:02.538703 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:02.539650 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:02.539651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:02.540206 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:02.540207 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:02.540557 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:02.540558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:02.540599 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:02.545863 (+ 5264us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:06:02.545868 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:06:02.547620 (+ 1752us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":123,"threads_started":1}
W20260925 22:06:02.547830 14330 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:06:02.538160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:51661 (local address 127.5.0.126:38597)
0925 22:06:02.538246 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:02.538252 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:02.538338 (+ 86us) server_negotiation.cc:407] Connection header received
0925 22:06:02.538405 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:02.538405 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:02.538417 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:02.538464 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:06:02.538785 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:02.539584 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:02.540296 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:02.540486 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:02.540552 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:02.547746 (+ 7194us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.66:51661: BlockingRecv error: recv got EOF from 127.5.0.66:51661 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 25 22:06:02 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:06:02 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.67@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:06:02.662277 14332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:02.657553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.67:36185)
0925 22:06:02.657724 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:02.657734 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:06:02.657792 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:06:02.657926 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:06:02.657927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:06:02.658083 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:06:02.658131 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:02.658133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:02.658967 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:02.658968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:02.659496 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:02.659498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:02.659857 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:02.659857 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:02.659946 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:02.660390 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:06:02.660395 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:06:02.662178 (+ 1783us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":100,"threads_started":1}
W20260925 22:06:02.662331 14330 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:06:02.657630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:36185 (local address 127.5.0.126:38597)
0925 22:06:02.657721 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:02.657723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:02.657794 (+ 71us) server_negotiation.cc:407] Connection header received
0925 22:06:02.657837 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:02.657837 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:02.657849 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:02.657909 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:06:02.658197 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:02.658891 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:02.659596 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:02.659783 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:02.659848 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:02.662256 (+ 2408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.67:36185: BlockingRecv error: recv got EOF from 127.5.0.67:36185 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 25 22:06:03 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:06:03 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.65@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:06:03.409369 14333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:03.404702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.65:47747)
0925 22:06:03.404848 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:03.404859 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:06:03.404914 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:06:03.405060 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:06:03.405061 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:06:03.405227 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:06:03.405276 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:03.405278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:03.405935 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:03.405936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:03.406478 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:03.406480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:03.406861 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:03.406862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:03.406926 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:03.407422 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:06:03.407426 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:06:03.409263 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":81,"threads_started":1}
W20260925 22:06:03.409432 14334 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:06:03.404797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:47747 (local address 127.5.0.126:38597)
0925 22:06:03.404938 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:03.404939 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:03.404948 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:06:03.404971 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:03.404971 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:03.404981 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:03.405041 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:06:03.405341 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:03.405853 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:03.406551 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:03.406761 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:03.406837 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:03.409359 (+ 2522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.65:47747: BlockingRecv error: recv got EOF from 127.5.0.65:47747 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
Sep 25 22:06:03 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:06:03 dist-test-slave-7mqr krb5kdc[13274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.0.66@KRBTEST.COM for oryx/127.5.0.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:06:03.553090 14335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:03.548350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:38597 (local address 127.5.0.66:40533)
0925 22:06:03.548516 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:03.548529 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:06:03.548624 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:06:03.548839 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:06:03.548839 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:06:03.549044 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:06:03.549096 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:03.549098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:03.549843 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:03.549845 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:03.550432 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:03.550434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:03.550745 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:03.550746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:03.550785 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:03.551235 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:06:03.551240 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:06:03.552998 (+ 1758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.0.126:38597: Server oryx/127.5.0.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":65,"threads_started":1}
W20260925 22:06:03.553293 14334 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:06:03.548414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:40533 (local address 127.5.0.126:38597)
0925 22:06:03.548469 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:03.548472 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:03.548628 (+ 156us) server_negotiation.cc:407] Connection header received
0925 22:06:03.548699 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:03.548700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:03.548728 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:03.548794 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:06:03.549182 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:03.549734 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:03.550501 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:03.550683 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:03.550734 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:03.553106 (+ 2372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.0.66:40533: BlockingRecv error: recv got EOF from 127.5.0.66:40533 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 22:06:03.580958 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 13857
I20260925 22:06:03.585322 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 14009
I20260925 22:06:03.589448 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 14143
I20260925 22:06:03.593530 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 13787
2026-09-25T22:06:03Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17395 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-25T22:06:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:06:03Z Disabled control of system clock
I20260925 22:06:03.620836 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:42609
--webserver_interface=127.5.0.126
--webserver_port=0
--builtin_ntp_servers=127.5.0.84:35685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:42609
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260925 22:06:03.700009 14339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:06:03.700237 14339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:06:03.700260 14339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:06:03.701440 14339 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:06:03.701478 14339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:06:03.701491 14339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:06:03.701504 14339 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:06:03.701519 14339 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:06:03.702809 14339 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:35685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:42609
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:42609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:06:03.703007 14339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:06:03.703244 14339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:06:03.708573 14347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:06:03.708585 14345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:06:03.708696 14344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:06:03.708588 14339 server_base.cc:1034] running on GCE node
I20260925 22:06:03.709095 14339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:06:03.709437 14339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:06:03.710603 14339 hybrid_clock.cc:648] HybridClock initialized: now 1790373963710592 us; error 33 us; skew 500 ppm
I20260925 22:06:03.711849 14339 webserver.cc:460] Webserver started at http://127.5.0.126:32787/ using document root <none> and password file <none>
I20260925 22:06:03.712118 14339 fs_manager.cc:359] Metadata directory not provided
I20260925 22:06:03.712203 14339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:06:03.712363 14339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:06:03.713424 14339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "e5f043a8d9304028a3b5f327e2c57cea"
format_stamp: "Formatted at 2026-09-25 22:06:03 on dist-test-slave-7mqr"
server_key: "afc4eb4d40dbe676627c64bfbd35f7a5"
server_key_iv: "67813b6c1311d9c0119f74af0f02fcae"
server_key_version: "encryptionkey@0"
I20260925 22:06:03.713786 14339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "e5f043a8d9304028a3b5f327e2c57cea"
format_stamp: "Formatted at 2026-09-25 22:06:03 on dist-test-slave-7mqr"
server_key: "afc4eb4d40dbe676627c64bfbd35f7a5"
server_key_iv: "67813b6c1311d9c0119f74af0f02fcae"
server_key_version: "encryptionkey@0"
I20260925 22:06:03.715071 14339 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:06:03.715782 14353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:03.715983 14339 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:06:03.716075 14339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "e5f043a8d9304028a3b5f327e2c57cea"
format_stamp: "Formatted at 2026-09-25 22:06:03 on dist-test-slave-7mqr"
server_key: "afc4eb4d40dbe676627c64bfbd35f7a5"
server_key_iv: "67813b6c1311d9c0119f74af0f02fcae"
server_key_version: "encryptionkey@0"
I20260925 22:06:03.716158 14339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:06:03.730815 14339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:06:03.731132 14339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:06:03.731283 14339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:06:03.735335 14339 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:42609
I20260925 22:06:03.735369 14405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:42609 every 8 connection(s)
I20260925 22:06:03.735707 14339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:06:03.736371 14406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:06:03.738963 14406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5f043a8d9304028a3b5f327e2c57cea: Bootstrap starting.
I20260925 22:06:03.739566 14406 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5f043a8d9304028a3b5f327e2c57cea: Neither blocks nor log segments found. Creating new log.
I20260925 22:06:03.739867 14406 log.cc:824] T 00000000000000000000000000000000 P e5f043a8d9304028a3b5f327e2c57cea: Log is configured to *not* fsync() on all Append() calls
I20260925 22:06:03.740545 14406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5f043a8d9304028a3b5f327e2c57cea: No bootstrap required, opened a new log
I20260925 22:06:03.742120 14406 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5f043a8d9304028a3b5f327e2c57cea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5f043a8d9304028a3b5f327e2c57cea" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42609 } }
I20260925 22:06:03.742252 14406 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5f043a8d9304028a3b5f327e2c57cea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:06:03.742273 14406 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5f043a8d9304028a3b5f327e2c57cea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5f043a8d9304028a3b5f327e2c57cea, State: Initialized, Role: FOLLOWER
I20260925 22:06:03.742452 14406 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5f043a8d9304028a3b5f327e2c57cea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5f043a8d9304028a3b5f327e2c57cea" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42609 } }
I20260925 22:06:03.742513 14406 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5f043a8d9304028a3b5f327e2c57cea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:06:03.742534 14406 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5f043a8d9304028a3b5f327e2c57cea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:06:03.742604 14406 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5f043a8d9304028a3b5f327e2c57cea [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:06:03.743113 14406 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5f043a8d9304028a3b5f327e2c57cea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5f043a8d9304028a3b5f327e2c57cea" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42609 } }
I20260925 22:06:03.743227 14406 leader_election.cc:302] T 00000000000000000000000000000000 P e5f043a8d9304028a3b5f327e2c57cea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5f043a8d9304028a3b5f327e2c57cea; no voters:
I20260925 22:06:03.743403 14406 leader_election.cc:288] T 00000000000000000000000000000000 P e5f043a8d9304028a3b5f327e2c57cea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:06:03.743470 14409 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5f043a8d9304028a3b5f327e2c57cea [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:06:03.743635 14409 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5f043a8d9304028a3b5f327e2c57cea [term 1 LEADER]: Becoming Leader. State: Replica: e5f043a8d9304028a3b5f327e2c57cea, State: Running, Role: LEADER
I20260925 22:06:03.743680 14406 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5f043a8d9304028a3b5f327e2c57cea [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:06:03.743745 14409 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5f043a8d9304028a3b5f327e2c57cea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5f043a8d9304028a3b5f327e2c57cea" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42609 } }
I20260925 22:06:03.744155 14411 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5f043a8d9304028a3b5f327e2c57cea [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5f043a8d9304028a3b5f327e2c57cea. Latest consensus state: current_term: 1 leader_uuid: "e5f043a8d9304028a3b5f327e2c57cea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5f043a8d9304028a3b5f327e2c57cea" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42609 } } }
I20260925 22:06:03.744232 14411 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5f043a8d9304028a3b5f327e2c57cea [sys.catalog]: This master's current role is: LEADER
I20260925 22:06:03.744309 14410 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5f043a8d9304028a3b5f327e2c57cea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5f043a8d9304028a3b5f327e2c57cea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5f043a8d9304028a3b5f327e2c57cea" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42609 } } }
I20260925 22:06:03.744390 14410 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5f043a8d9304028a3b5f327e2c57cea [sys.catalog]: This master's current role is: LEADER
I20260925 22:06:03.744719 14417 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:06:03.744972 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 14339
I20260925 22:06:03.745088 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:06:03.745188 14417 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:06:03.745292 5121 external_mini_cluster.cc:1428] Setting key 85eec1676af1cc5c48564e95971fdd8f
I20260925 22:06:03.746778 14417 catalog_manager.cc:1347] Generated new cluster ID: 74e57e265858484798c4f350293cdd72
I20260925 22:06:03.746826 14417 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:06:03.751737 14428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:03.746022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48214 (local address 127.5.0.126:42609)
0925 22:06:03.746206 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:03.746215 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:03.746269 (+ 54us) server_negotiation.cc:407] Connection header received
0925 22:06:03.746584 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:03.746592 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:03.746670 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:03.746809 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:06:03.747038 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:03.748376 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:03.749471 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:03.750056 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:03.750157 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:03.750876 (+ 719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:06:03.750879 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:06:03.750880 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:06:03.751076 (+ 196us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:06:03.751113 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:06:03.751125 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:06:03.751186 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 22:06:03.751284 (+ 98us) server_negotiation.cc:299] Negotiation successful
0925 22:06:03.751355 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":68,"threads_started":1}
I20260925 22:06:03.756867 14417 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:06:03.757334 14417 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:06:03.763702 14417 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5f043a8d9304028a3b5f327e2c57cea: Generated new TSK 0
I20260925 22:06:03.763875 14417 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:06:03.767771 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.65:0
--local_ip_for_outbound_sockets=127.5.0.65
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:42609
--builtin_ntp_servers=127.5.0.84:35685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260925 22:06:03.848215 14430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:06:03.848425 14430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:06:03.848446 14430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:06:03.849601 14430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:06:03.849642 14430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:06:03.849670 14430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.65
I20260925 22:06:03.851042 14430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:35685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:42609
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.65
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:06:03.851289 14430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:06:03.851583 14430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:06:03.857313 14435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:06:03.857460 14430 server_base.cc:1034] running on GCE node
W20260925 22:06:03.857296 14436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:06:03.857283 14438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:06:03.857812 14430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:06:03.858103 14430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:06:03.859272 14430 hybrid_clock.cc:648] HybridClock initialized: now 1790373963859256 us; error 31 us; skew 500 ppm
I20260925 22:06:03.860688 14430 webserver.cc:460] Webserver started at http://127.5.0.65:37397/ using document root <none> and password file <none>
I20260925 22:06:03.860946 14430 fs_manager.cc:359] Metadata directory not provided
I20260925 22:06:03.861012 14430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:06:03.861159 14430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:06:03.862262 14430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/data/instance:
uuid: "740828a89f6f438292c2f9b09e3de4c8"
format_stamp: "Formatted at 2026-09-25 22:06:03 on dist-test-slave-7mqr"
server_key: "225039b9a8955e05ffeb9106e9278166"
server_key_iv: "8b1db92fe0cc2bb13cb87038ed33fc00"
server_key_version: "encryptionkey@0"
I20260925 22:06:03.862619 14430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance:
uuid: "740828a89f6f438292c2f9b09e3de4c8"
format_stamp: "Formatted at 2026-09-25 22:06:03 on dist-test-slave-7mqr"
server_key: "225039b9a8955e05ffeb9106e9278166"
server_key_iv: "8b1db92fe0cc2bb13cb87038ed33fc00"
server_key_version: "encryptionkey@0"
I20260925 22:06:03.863981 14430 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:06:03.864792 14444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:03.864992 14430 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:06:03.865090 14430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/wal
uuid: "740828a89f6f438292c2f9b09e3de4c8"
format_stamp: "Formatted at 2026-09-25 22:06:03 on dist-test-slave-7mqr"
server_key: "225039b9a8955e05ffeb9106e9278166"
server_key_iv: "8b1db92fe0cc2bb13cb87038ed33fc00"
server_key_version: "encryptionkey@0"
I20260925 22:06:03.865178 14430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:06:03.879779 14430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:06:03.880177 14430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:06:03.880347 14430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:06:03.880604 14430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:06:03.880954 14430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:06:03.880988 14430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:03.881047 14430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:06:03.881068 14430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:03.886960 14430 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.65:36175
I20260925 22:06:03.887039 14557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.65:36175 every 8 connection(s)
I20260925 22:06:03.887388 14430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
I20260925 22:06:03.892601 14428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:03.888173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:59289 (local address 127.5.0.126:42609)
0925 22:06:03.888245 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:03.888248 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:03.888462 (+ 214us) server_negotiation.cc:407] Connection header received
0925 22:06:03.888982 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:03.888983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:03.888997 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:03.889062 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:06:03.889702 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:03.890209 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:03.891182 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:03.891365 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:03.891418 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:03.892321 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:06:03.892322 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:06:03.892322 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:06:03.892326 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:06:03.892343 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:06:03.892347 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:06:03.892391 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 22:06:03.892473 (+ 82us) server_negotiation.cc:299] Negotiation successful
0925 22:06:03.892498 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 22:06:03.892776 14560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:03.888177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42609 (local address 127.5.0.65:59289)
0925 22:06:03.888352 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:03.888388 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 22:06:03.888890 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:06:03.889119 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:06:03.889122 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:06:03.889130 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:06:03.889625 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:03.889629 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:03.890297 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:03.890298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:03.891099 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:03.891101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:03.891473 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:03.891474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:03.891606 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:03.892188 (+ 582us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:06:03.892189 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:06:03.892218 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:06:03.892240 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:06:03.892407 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:06:03.892410 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:06:03.892412 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 22:06:03.892463 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 22:06:03.892543 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":75,"threads_started":1}
I20260925 22:06:03.892776 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 14430
I20260925 22:06:03.892874 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance
I20260925 22:06:03.893072 5121 external_mini_cluster.cc:1428] Setting key 087a139382bf742fd5c1bb2cc30dab4c
I20260925 22:06:03.893333 14558 heartbeater.cc:347] Connected to a master server at 127.5.0.126:42609
I20260925 22:06:03.893448 14558 heartbeater.cc:464] Registering TS with master...
I20260925 22:06:03.893659 14558 heartbeater.cc:511] Master 127.5.0.126:42609 requested a full tablet report, sending...
I20260925 22:06:03.894120 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.66:0
--local_ip_for_outbound_sockets=127.5.0.66
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:42609
--builtin_ntp_servers=127.5.0.84:35685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260925 22:06:03.894168 14370 ts_manager.cc:194] Registered new tserver with Master: 740828a89f6f438292c2f9b09e3de4c8 (127.5.0.65:36175)
I20260925 22:06:03.894779 14370 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.0.65:59289
W20260925 22:06:03.975804 14561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:06:03.976037 14561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:06:03.976058 14561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:06:03.977241 14561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:06:03.977279 14561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:06:03.977303 14561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.66
I20260925 22:06:03.978662 14561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:35685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:42609
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.66
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:06:03.978855 14561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:06:03.979100 14561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:06:03.984141 14569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:06:03.984149 14567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:06:03.984149 14566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:06:03.984266 14561 server_base.cc:1034] running on GCE node
I20260925 22:06:03.984611 14561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:06:03.984913 14561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:06:03.986084 14561 hybrid_clock.cc:648] HybridClock initialized: now 1790373963986064 us; error 54 us; skew 500 ppm
I20260925 22:06:03.987465 14561 webserver.cc:460] Webserver started at http://127.5.0.66:37589/ using document root <none> and password file <none>
I20260925 22:06:03.987709 14561 fs_manager.cc:359] Metadata directory not provided
I20260925 22:06:03.987780 14561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:06:03.987957 14561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:06:03.989140 14561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/data/instance:
uuid: "13de0fff6f7449988991363f28f36aa4"
format_stamp: "Formatted at 2026-09-25 22:06:03 on dist-test-slave-7mqr"
server_key: "3416db9a1235fdab19f58cffd469f79a"
server_key_iv: "f3ca6f37e62ce9db9685a8e8369430c6"
server_key_version: "encryptionkey@0"
I20260925 22:06:03.989511 14561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance:
uuid: "13de0fff6f7449988991363f28f36aa4"
format_stamp: "Formatted at 2026-09-25 22:06:03 on dist-test-slave-7mqr"
server_key: "3416db9a1235fdab19f58cffd469f79a"
server_key_iv: "f3ca6f37e62ce9db9685a8e8369430c6"
server_key_version: "encryptionkey@0"
I20260925 22:06:03.990813 14561 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:06:03.991488 14575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:03.991654 14561 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:06:03.991755 14561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/wal
uuid: "13de0fff6f7449988991363f28f36aa4"
format_stamp: "Formatted at 2026-09-25 22:06:03 on dist-test-slave-7mqr"
server_key: "3416db9a1235fdab19f58cffd469f79a"
server_key_iv: "f3ca6f37e62ce9db9685a8e8369430c6"
server_key_version: "encryptionkey@0"
I20260925 22:06:03.991827 14561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:06:04.011586 14561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:06:04.011992 14561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:06:04.012151 14561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:06:04.012398 14561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:06:04.012746 14561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:06:04.012801 14561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:04.012844 14561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:06:04.012885 14561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:04.018853 14561 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.66:38213
I20260925 22:06:04.019006 14688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.66:38213 every 8 connection(s)
I20260925 22:06:04.019289 14561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
I20260925 22:06:04.025250 14428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:04.020370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:53743 (local address 127.5.0.126:42609)
0925 22:06:04.020413 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:04.020415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:04.020472 (+ 57us) server_negotiation.cc:407] Connection header received
0925 22:06:04.021329 (+ 857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:04.021330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:04.021349 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:04.021432 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:06:04.022288 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:04.022817 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:04.023826 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:04.024104 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:04.024176 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:04.024958 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:06:04.024960 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:06:04.024960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:06:04.024965 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:06:04.024979 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:06:04.024983 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:06:04.025030 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 22:06:04.025138 (+ 108us) server_negotiation.cc:299] Negotiation successful
0925 22:06:04.025169 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 22:06:04.025472 14691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:04.020121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42609 (local address 127.5.0.66:53743)
0925 22:06:04.020341 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:04.020390 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 22:06:04.021053 (+ 663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:06:04.021576 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:06:04.021581 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:06:04.021593 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:06:04.022185 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:04.022191 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:04.022915 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:04.022916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:04.023716 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:04.023719 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:04.024194 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:04.024195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:04.024327 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:04.024808 (+ 481us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:06:04.024808 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:06:04.024836 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:06:04.024859 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:06:04.025029 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:06:04.025034 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:06:04.025038 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:06:04.025140 (+ 102us) client_negotiation.cc:240] Negotiation successful
0925 22:06:04.025241 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":87,"threads_started":1}
I20260925 22:06:04.025942 14689 heartbeater.cc:347] Connected to a master server at 127.5.0.126:42609
I20260925 22:06:04.026049 14689 heartbeater.cc:464] Registering TS with master...
I20260925 22:06:04.026233 14689 heartbeater.cc:511] Master 127.5.0.126:42609 requested a full tablet report, sending...
I20260925 22:06:04.026636 14370 ts_manager.cc:194] Registered new tserver with Master: 13de0fff6f7449988991363f28f36aa4 (127.5.0.66:38213)
I20260925 22:06:04.027000 14370 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.0.66:53743
I20260925 22:06:04.029155 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 14561
I20260925 22:06:04.029271 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance
I20260925 22:06:04.029469 5121 external_mini_cluster.cc:1428] Setting key 1e3cf1b0381fd78133dfa6d5fe43ddb0
I20260925 22:06:04.030315 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.67:0
--local_ip_for_outbound_sockets=127.5.0.67
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:42609
--builtin_ntp_servers=127.5.0.84:35685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260925 22:06:04.113945 14692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:06:04.114178 14692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:06:04.114216 14692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:06:04.115322 14692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:06:04.115387 14692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:06:04.115427 14692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.67
I20260925 22:06:04.116776 14692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:35685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:42609
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.67
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:06:04.117033 14692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:06:04.117321 14692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:06:04.122253 14700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:06:04.122262 14698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:06:04.122319 14692 server_base.cc:1034] running on GCE node
W20260925 22:06:04.122272 14697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:06:04.122658 14692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:06:04.122896 14692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:06:04.124059 14692 hybrid_clock.cc:648] HybridClock initialized: now 1790373964124042 us; error 39 us; skew 500 ppm
I20260925 22:06:04.125252 14692 webserver.cc:460] Webserver started at http://127.5.0.67:46413/ using document root <none> and password file <none>
I20260925 22:06:04.125483 14692 fs_manager.cc:359] Metadata directory not provided
I20260925 22:06:04.125556 14692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:06:04.125713 14692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:06:04.126806 14692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/data/instance:
uuid: "ac1741e0a8b84de9a643ff99f9256e0c"
format_stamp: "Formatted at 2026-09-25 22:06:04 on dist-test-slave-7mqr"
server_key: "11be06d2e8f05876a8674ebcd8a3f078"
server_key_iv: "68c0bb5ce241813e3d2dd9a49e264ba0"
server_key_version: "encryptionkey@0"
I20260925 22:06:04.127163 14692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance:
uuid: "ac1741e0a8b84de9a643ff99f9256e0c"
format_stamp: "Formatted at 2026-09-25 22:06:04 on dist-test-slave-7mqr"
server_key: "11be06d2e8f05876a8674ebcd8a3f078"
server_key_iv: "68c0bb5ce241813e3d2dd9a49e264ba0"
server_key_version: "encryptionkey@0"
I20260925 22:06:04.128495 14692 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:06:04.129186 14706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:04.129343 14692 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:06:04.129438 14692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/wal
uuid: "ac1741e0a8b84de9a643ff99f9256e0c"
format_stamp: "Formatted at 2026-09-25 22:06:04 on dist-test-slave-7mqr"
server_key: "11be06d2e8f05876a8674ebcd8a3f078"
server_key_iv: "68c0bb5ce241813e3d2dd9a49e264ba0"
server_key_version: "encryptionkey@0"
I20260925 22:06:04.129525 14692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:06:04.142489 14692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:06:04.142776 14692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:06:04.142920 14692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:06:04.143152 14692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:06:04.143451 14692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:06:04.143504 14692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:04.143571 14692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:06:04.143595 14692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:04.149390 14692 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.67:35395
I20260925 22:06:04.149427 14819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.67:35395 every 8 connection(s)
I20260925 22:06:04.149742 14692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
I20260925 22:06:04.154616 14428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:04.150461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:56627 (local address 127.5.0.126:42609)
0925 22:06:04.150548 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:04.150551 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:04.150776 (+ 225us) server_negotiation.cc:407] Connection header received
0925 22:06:04.151227 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:04.151228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:04.151238 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:04.151326 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:06:04.151922 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:04.152422 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:04.153341 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:04.153544 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:04.153596 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:04.154422 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:06:04.154423 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:06:04.154423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:06:04.154427 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:06:04.154438 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:06:04.154442 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:06:04.154484 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0925 22:06:04.154536 (+ 52us) server_negotiation.cc:299] Negotiation successful
0925 22:06:04.154557 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 22:06:04.154829 14822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:04.150448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42609 (local address 127.5.0.67:56627)
0925 22:06:04.150651 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:04.150692 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 22:06:04.151144 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:06:04.151344 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:06:04.151347 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:06:04.151355 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:06:04.151833 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:04.151837 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:04.152500 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:04.152501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:04.153269 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:04.153271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:04.153615 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:04.153616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:04.153727 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:04.154298 (+ 571us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:06:04.154298 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:06:04.154325 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:06:04.154346 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:06:04.154485 (+ 139us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:06:04.154487 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:06:04.154489 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 22:06:04.154534 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 22:06:04.154606 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":102,"threads_started":1}
I20260925 22:06:04.155298 14820 heartbeater.cc:347] Connected to a master server at 127.5.0.126:42609
I20260925 22:06:04.155372 14820 heartbeater.cc:464] Registering TS with master...
I20260925 22:06:04.155586 14820 heartbeater.cc:511] Master 127.5.0.126:42609 requested a full tablet report, sending...
I20260925 22:06:04.155635 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 14692
I20260925 22:06:04.155740 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance
I20260925 22:06:04.155990 5121 external_mini_cluster.cc:1428] Setting key 3b942cf8c2da725c824d6496f289da52
I20260925 22:06:04.155987 14370 ts_manager.cc:194] Registered new tserver with Master: ac1741e0a8b84de9a643ff99f9256e0c (127.5.0.67:35395)
I20260925 22:06:04.156489 14370 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.0.67:56627
I20260925 22:06:04.156888 5121 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 22:06:04.158543 14428 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:06:04.158184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48222 (local address 127.5.0.126:42609)
0925 22:06:04.158220 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:04.158222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:04.158318 (+ 96us) server_negotiation.cc:407] Connection header received
0925 22:06:04.158360 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:04.158360 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:04.158370 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:04.158415 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:06:04.158502 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48222: BlockingRecv error: recv got EOF from 127.0.0.1:48222 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 22:06:04.158679 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 14430
I20260925 22:06:04.163628 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 14561
I20260925 22:06:04.167701 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 14692
I20260925 22:06:04.171584 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 14339
2026-09-25T22:06:04Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (576 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-25T22:06:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:06:04Z Disabled control of system clock
I20260925 22:06:04.197593 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:36971
--webserver_interface=127.5.0.126
--webserver_port=0
--builtin_ntp_servers=127.5.0.84:44651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:36971
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 22:06:04.278119 14831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:06:04.278331 14831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:06:04.278353 14831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:06:04.279505 14831 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:06:04.279551 14831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:06:04.279563 14831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:06:04.279577 14831 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:06:04.279633 14831 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:06:04.281086 14831 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:44651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:36971
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:36971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:06:04.281334 14831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:06:04.281621 14831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:06:04.286332 14836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:06:04.286348 14839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:06:04.286373 14837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:06:04.286429 14831 server_base.cc:1034] running on GCE node
I20260925 22:06:04.286811 14831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:06:04.287134 14831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:06:04.288290 14831 hybrid_clock.cc:648] HybridClock initialized: now 1790373964288265 us; error 36 us; skew 500 ppm
I20260925 22:06:04.289475 14831 webserver.cc:460] Webserver started at http://127.5.0.126:37087/ using document root <none> and password file <none>
I20260925 22:06:04.289742 14831 fs_manager.cc:359] Metadata directory not provided
I20260925 22:06:04.289808 14831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:06:04.289953 14831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:06:04.291121 14831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "7e8227cd482e4af08d74fad5df56bd45"
format_stamp: "Formatted at 2026-09-25 22:06:04 on dist-test-slave-7mqr"
server_key: "d0dcc15f31721e5f28dd3339ba8c5504"
server_key_iv: "32b710dfa6f8f1dc94defe1e86d0d6d9"
server_key_version: "encryptionkey@0"
I20260925 22:06:04.291509 14831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "7e8227cd482e4af08d74fad5df56bd45"
format_stamp: "Formatted at 2026-09-25 22:06:04 on dist-test-slave-7mqr"
server_key: "d0dcc15f31721e5f28dd3339ba8c5504"
server_key_iv: "32b710dfa6f8f1dc94defe1e86d0d6d9"
server_key_version: "encryptionkey@0"
I20260925 22:06:04.292915 14831 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:06:04.293742 14845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:04.293916 14831 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:06:04.294037 14831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "7e8227cd482e4af08d74fad5df56bd45"
format_stamp: "Formatted at 2026-09-25 22:06:04 on dist-test-slave-7mqr"
server_key: "d0dcc15f31721e5f28dd3339ba8c5504"
server_key_iv: "32b710dfa6f8f1dc94defe1e86d0d6d9"
server_key_version: "encryptionkey@0"
I20260925 22:06:04.294112 14831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:06:04.297504 14831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:06:04.297771 14831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:06:04.297901 14831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:06:04.301760 14831 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:36971
I20260925 22:06:04.301790 14897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:36971 every 8 connection(s)
I20260925 22:06:04.302110 14831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:06:04.302736 14898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:06:04.305408 14898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e8227cd482e4af08d74fad5df56bd45: Bootstrap starting.
I20260925 22:06:04.305912 14898 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e8227cd482e4af08d74fad5df56bd45: Neither blocks nor log segments found. Creating new log.
I20260925 22:06:04.306150 14898 log.cc:824] T 00000000000000000000000000000000 P 7e8227cd482e4af08d74fad5df56bd45: Log is configured to *not* fsync() on all Append() calls
I20260925 22:06:04.306770 14898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e8227cd482e4af08d74fad5df56bd45: No bootstrap required, opened a new log
I20260925 22:06:04.308339 14898 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e8227cd482e4af08d74fad5df56bd45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8227cd482e4af08d74fad5df56bd45" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 36971 } }
I20260925 22:06:04.308465 14898 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e8227cd482e4af08d74fad5df56bd45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:06:04.308486 14898 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e8227cd482e4af08d74fad5df56bd45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e8227cd482e4af08d74fad5df56bd45, State: Initialized, Role: FOLLOWER
I20260925 22:06:04.308581 14898 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e8227cd482e4af08d74fad5df56bd45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8227cd482e4af08d74fad5df56bd45" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 36971 } }
I20260925 22:06:04.308658 14898 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e8227cd482e4af08d74fad5df56bd45 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:06:04.308691 14898 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e8227cd482e4af08d74fad5df56bd45 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:06:04.308759 14898 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e8227cd482e4af08d74fad5df56bd45 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:06:04.309279 14898 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e8227cd482e4af08d74fad5df56bd45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8227cd482e4af08d74fad5df56bd45" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 36971 } }
I20260925 22:06:04.309391 14898 leader_election.cc:302] T 00000000000000000000000000000000 P 7e8227cd482e4af08d74fad5df56bd45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e8227cd482e4af08d74fad5df56bd45; no voters:
I20260925 22:06:04.309552 14898 leader_election.cc:288] T 00000000000000000000000000000000 P 7e8227cd482e4af08d74fad5df56bd45 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:06:04.309598 14901 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e8227cd482e4af08d74fad5df56bd45 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:06:04.309793 14898 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e8227cd482e4af08d74fad5df56bd45 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:06:04.309791 14901 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e8227cd482e4af08d74fad5df56bd45 [term 1 LEADER]: Becoming Leader. State: Replica: 7e8227cd482e4af08d74fad5df56bd45, State: Running, Role: LEADER
I20260925 22:06:04.309914 14901 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e8227cd482e4af08d74fad5df56bd45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8227cd482e4af08d74fad5df56bd45" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 36971 } }
I20260925 22:06:04.310513 14903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e8227cd482e4af08d74fad5df56bd45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e8227cd482e4af08d74fad5df56bd45. Latest consensus state: current_term: 1 leader_uuid: "7e8227cd482e4af08d74fad5df56bd45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8227cd482e4af08d74fad5df56bd45" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 36971 } } }
I20260925 22:06:04.310598 14903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e8227cd482e4af08d74fad5df56bd45 [sys.catalog]: This master's current role is: LEADER
I20260925 22:06:04.310873 14902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e8227cd482e4af08d74fad5df56bd45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e8227cd482e4af08d74fad5df56bd45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8227cd482e4af08d74fad5df56bd45" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 36971 } } }
I20260925 22:06:04.310917 14910 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:06:04.310945 14902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e8227cd482e4af08d74fad5df56bd45 [sys.catalog]: This master's current role is: LEADER
I20260925 22:06:04.311450 14910 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:06:04.311546 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 14831
I20260925 22:06:04.311640 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:06:04.311841 5121 external_mini_cluster.cc:1428] Setting key faf6eb751b58347502f7191390a67f2e
I20260925 22:06:04.313069 14910 catalog_manager.cc:1347] Generated new cluster ID: 5ffe61bc4746454fbb4dc01b1be69de2
I20260925 22:06:04.313122 14910 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:06:04.314411 14920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:04.312293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42622 (local address 127.5.0.126:36971)
0925 22:06:04.312472 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:04.312480 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:04.312734 (+ 254us) server_negotiation.cc:407] Connection header received
0925 22:06:04.312867 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:04.312874 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:04.312916 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:04.313050 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:06:04.313857 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:06:04.313861 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:06:04.313861 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:06:04.313865 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:06:04.313907 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:06:04.313921 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:06:04.313973 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 22:06:04.314190 (+ 217us) server_negotiation.cc:299] Negotiation successful
0925 22:06:04.314244 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":85,"threads_started":1}
I20260925 22:06:04.329535 14910 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:06:04.329715 14910 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:06:04.332068 14910 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e8227cd482e4af08d74fad5df56bd45: Generated new TSK 0
I20260925 22:06:04.332267 14910 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:06:04.337951 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.65:0
--local_ip_for_outbound_sockets=127.5.0.65
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:36971
--builtin_ntp_servers=127.5.0.84:44651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 22:06:04.418812 14922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:06:04.419006 14922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:06:04.419027 14922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:06:04.420270 14922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:06:04.420310 14922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:06:04.420336 14922 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.65
I20260925 22:06:04.421662 14922 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:44651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:36971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.65
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:06:04.421896 14922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:06:04.422240 14922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:06:04.427078 14928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:06:04.427129 14927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:06:04.427217 14922 server_base.cc:1034] running on GCE node
W20260925 22:06:04.427412 14930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:06:04.427639 14922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:06:04.427917 14922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:06:04.429083 14922 hybrid_clock.cc:648] HybridClock initialized: now 1790373964429070 us; error 28 us; skew 500 ppm
I20260925 22:06:04.430457 14922 webserver.cc:460] Webserver started at http://127.5.0.65:43733/ using document root <none> and password file <none>
I20260925 22:06:04.430711 14922 fs_manager.cc:359] Metadata directory not provided
I20260925 22:06:04.430765 14922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:06:04.430891 14922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:06:04.432216 14922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/data/instance:
uuid: "d1b2df883e164070bfc4a324d304633b"
format_stamp: "Formatted at 2026-09-25 22:06:04 on dist-test-slave-7mqr"
server_key: "57842243e8c32fccfba7d9b86aaefbcc"
server_key_iv: "86d227952b34924326c87892993fe57c"
server_key_version: "encryptionkey@0"
I20260925 22:06:04.432617 14922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance:
uuid: "d1b2df883e164070bfc4a324d304633b"
format_stamp: "Formatted at 2026-09-25 22:06:04 on dist-test-slave-7mqr"
server_key: "57842243e8c32fccfba7d9b86aaefbcc"
server_key_iv: "86d227952b34924326c87892993fe57c"
server_key_version: "encryptionkey@0"
I20260925 22:06:04.433964 14922 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:06:04.434612 14936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:04.434803 14922 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:06:04.434880 14922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/wal
uuid: "d1b2df883e164070bfc4a324d304633b"
format_stamp: "Formatted at 2026-09-25 22:06:04 on dist-test-slave-7mqr"
server_key: "57842243e8c32fccfba7d9b86aaefbcc"
server_key_iv: "86d227952b34924326c87892993fe57c"
server_key_version: "encryptionkey@0"
I20260925 22:06:04.434932 14922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:06:04.438215 14922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:06:04.438426 14922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:06:04.438524 14922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:06:04.438678 14922 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:06:04.438906 14922 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:06:04.438935 14922 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:04.438954 14922 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:06:04.438967 14922 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:04.444478 14922 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.65:42039
I20260925 22:06:04.444558 15049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.65:42039 every 8 connection(s)
I20260925 22:06:04.444880 14922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
I20260925 22:06:04.447755 14920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:04.445640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:33955 (local address 127.5.0.126:36971)
0925 22:06:04.445683 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:04.445686 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:04.445927 (+ 241us) server_negotiation.cc:407] Connection header received
0925 22:06:04.446368 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:04.446369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:04.446376 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:04.446437 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:06:04.447534 (+ 1097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:06:04.447535 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:06:04.447535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:06:04.447539 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:06:04.447554 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:06:04.447558 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:06:04.447607 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 22:06:04.447688 (+ 81us) server_negotiation.cc:299] Negotiation successful
0925 22:06:04.447703 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 22:06:04.447947 15052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:04.445582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:36971 (local address 127.5.0.65:33955)
0925 22:06:04.445795 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:04.445835 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 22:06:04.446285 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:06:04.446504 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:06:04.446507 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:06:04.446515 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:06:04.447381 (+ 866us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:06:04.447382 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:06:04.447409 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:06:04.447436 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:06:04.447619 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:06:04.447623 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:06:04.447626 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:06:04.447685 (+ 59us) client_negotiation.cc:240] Negotiation successful
0925 22:06:04.447733 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":112,"threads_started":1}
I20260925 22:06:04.448422 15050 heartbeater.cc:347] Connected to a master server at 127.5.0.126:36971
I20260925 22:06:04.448523 15050 heartbeater.cc:464] Registering TS with master...
I20260925 22:06:04.448673 15050 heartbeater.cc:511] Master 127.5.0.126:36971 requested a full tablet report, sending...
I20260925 22:06:04.449183 14862 ts_manager.cc:194] Registered new tserver with Master: d1b2df883e164070bfc4a324d304633b (127.5.0.65:42039)
I20260925 22:06:04.452664 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 14922
I20260925 22:06:04.452773 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance
I20260925 22:06:04.452958 5121 external_mini_cluster.cc:1428] Setting key 7dae0869c2e905e6d18df3924084d1e6
I20260925 22:06:04.453835 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.66:0
--local_ip_for_outbound_sockets=127.5.0.66
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:36971
--builtin_ntp_servers=127.5.0.84:44651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 22:06:04.532639 15053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:06:04.532855 15053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:06:04.532876 15053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:06:04.534029 15053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:06:04.534067 15053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:06:04.534092 15053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.66
I20260925 22:06:04.535408 15053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:44651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:36971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.66
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:06:04.535655 15053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:06:04.535960 15053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:06:04.540866 15058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:06:04.540866 15061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:06:04.540902 15059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:06:04.540892 15053 server_base.cc:1034] running on GCE node
I20260925 22:06:04.541280 15053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:06:04.541545 15053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:06:04.542706 15053 hybrid_clock.cc:648] HybridClock initialized: now 1790373964542692 us; error 34 us; skew 500 ppm
I20260925 22:06:04.543977 15053 webserver.cc:460] Webserver started at http://127.5.0.66:41469/ using document root <none> and password file <none>
I20260925 22:06:04.544209 15053 fs_manager.cc:359] Metadata directory not provided
I20260925 22:06:04.544252 15053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:06:04.544395 15053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:06:04.545409 15053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/data/instance:
uuid: "55fbf674b1a042b28ddb1ac822ab0e95"
format_stamp: "Formatted at 2026-09-25 22:06:04 on dist-test-slave-7mqr"
server_key: "ee6daeb2946ffface0b50d0c2982e265"
server_key_iv: "60fd28d081c3701a071da3033299bdf8"
server_key_version: "encryptionkey@0"
I20260925 22:06:04.545764 15053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance:
uuid: "55fbf674b1a042b28ddb1ac822ab0e95"
format_stamp: "Formatted at 2026-09-25 22:06:04 on dist-test-slave-7mqr"
server_key: "ee6daeb2946ffface0b50d0c2982e265"
server_key_iv: "60fd28d081c3701a071da3033299bdf8"
server_key_version: "encryptionkey@0"
I20260925 22:06:04.547037 15053 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:06:04.547787 15067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:04.547968 15053 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:06:04.548046 15053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/wal
uuid: "55fbf674b1a042b28ddb1ac822ab0e95"
format_stamp: "Formatted at 2026-09-25 22:06:04 on dist-test-slave-7mqr"
server_key: "ee6daeb2946ffface0b50d0c2982e265"
server_key_iv: "60fd28d081c3701a071da3033299bdf8"
server_key_version: "encryptionkey@0"
I20260925 22:06:04.548095 15053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:06:04.551288 15053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:06:04.551488 15053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:06:04.551578 15053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:06:04.551721 15053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:06:04.552063 15053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:06:04.552117 15053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:04.552157 15053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:06:04.552186 15053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:04.557679 15053 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.66:40837
I20260925 22:06:04.557716 15180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.66:40837 every 8 connection(s)
I20260925 22:06:04.558014 15053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
I20260925 22:06:04.560672 14920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:04.558666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:52763 (local address 127.5.0.126:36971)
0925 22:06:04.558734 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:04.558736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:04.558948 (+ 212us) server_negotiation.cc:407] Connection header received
0925 22:06:04.559352 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:04.559353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:04.559359 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:04.559411 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:06:04.560477 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:06:04.560477 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:06:04.560478 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:06:04.560481 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:06:04.560496 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:06:04.560500 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:06:04.560563 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 22:06:04.560607 (+ 44us) server_negotiation.cc:299] Negotiation successful
0925 22:06:04.560619 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 22:06:04.560907 15183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:04.558653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:36971 (local address 127.5.0.66:52763)
0925 22:06:04.558833 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:04.558864 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 22:06:04.559272 (+ 408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:06:04.559459 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:06:04.559462 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:06:04.559470 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:06:04.560341 (+ 871us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:06:04.560342 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:06:04.560368 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:06:04.560392 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:06:04.560559 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:06:04.560561 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:06:04.560564 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:06:04.560614 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 22:06:04.560658 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":89,"threads_started":1}
I20260925 22:06:04.561399 15181 heartbeater.cc:347] Connected to a master server at 127.5.0.126:36971
I20260925 22:06:04.561470 15181 heartbeater.cc:464] Registering TS with master...
I20260925 22:06:04.561627 15181 heartbeater.cc:511] Master 127.5.0.126:36971 requested a full tablet report, sending...
I20260925 22:06:04.561995 14862 ts_manager.cc:194] Registered new tserver with Master: 55fbf674b1a042b28ddb1ac822ab0e95 (127.5.0.66:40837)
I20260925 22:06:04.568085 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 15053
I20260925 22:06:04.568205 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance
I20260925 22:06:04.568390 5121 external_mini_cluster.cc:1428] Setting key c4478498be45d586ca9f272603a8c84f
I20260925 22:06:04.569286 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.67:0
--local_ip_for_outbound_sockets=127.5.0.67
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:36971
--builtin_ntp_servers=127.5.0.84:44651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 22:06:04.649688 15184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:06:04.649914 15184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:06:04.649951 15184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:06:04.651152 15184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:06:04.651214 15184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:06:04.651258 15184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.67
I20260925 22:06:04.652724 15184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:44651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:36971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.67
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:06:04.652989 15184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:06:04.653275 15184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:06:04.658048 15190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:06:04.658098 15189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:06:04.658097 15184 server_base.cc:1034] running on GCE node
W20260925 22:06:04.658038 15192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:06:04.658390 15184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:06:04.658587 15184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:06:04.659720 15184 hybrid_clock.cc:648] HybridClock initialized: now 1790373964659710 us; error 25 us; skew 500 ppm
I20260925 22:06:04.660928 15184 webserver.cc:460] Webserver started at http://127.5.0.67:39309/ using document root <none> and password file <none>
I20260925 22:06:04.661170 15184 fs_manager.cc:359] Metadata directory not provided
I20260925 22:06:04.661216 15184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:06:04.661377 15184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:06:04.662405 15184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/data/instance:
uuid: "c367411630be4e28be09afbad9b223f3"
format_stamp: "Formatted at 2026-09-25 22:06:04 on dist-test-slave-7mqr"
server_key: "ff0b096225e28e07420fa087a27c9ead"
server_key_iv: "6fcd65ba5f6703f93fdd2b13649946f7"
server_key_version: "encryptionkey@0"
I20260925 22:06:04.662755 15184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance:
uuid: "c367411630be4e28be09afbad9b223f3"
format_stamp: "Formatted at 2026-09-25 22:06:04 on dist-test-slave-7mqr"
server_key: "ff0b096225e28e07420fa087a27c9ead"
server_key_iv: "6fcd65ba5f6703f93fdd2b13649946f7"
server_key_version: "encryptionkey@0"
I20260925 22:06:04.664085 15184 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:06:04.664772 15198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:04.664952 15184 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:06:04.665056 15184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/wal
uuid: "c367411630be4e28be09afbad9b223f3"
format_stamp: "Formatted at 2026-09-25 22:06:04 on dist-test-slave-7mqr"
server_key: "ff0b096225e28e07420fa087a27c9ead"
server_key_iv: "6fcd65ba5f6703f93fdd2b13649946f7"
server_key_version: "encryptionkey@0"
I20260925 22:06:04.665126 15184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:06:04.668469 15184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:06:04.668711 15184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:06:04.668841 15184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:06:04.669035 15184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:06:04.669325 15184 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:06:04.669358 15184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:04.669377 15184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:06:04.669423 15184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:04.674829 15184 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.67:36343
I20260925 22:06:04.674901 15311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.67:36343 every 8 connection(s)
I20260925 22:06:04.675210 15184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
I20260925 22:06:04.677973 14920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:04.675929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:54349 (local address 127.5.0.126:36971)
0925 22:06:04.675979 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:04.675982 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:04.676221 (+ 239us) server_negotiation.cc:407] Connection header received
0925 22:06:04.676656 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:04.676657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:04.676663 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:04.676715 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:06:04.677780 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:06:04.677781 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:06:04.677782 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:06:04.677786 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:06:04.677801 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:06:04.677805 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:06:04.677861 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 22:06:04.677908 (+ 47us) server_negotiation.cc:299] Negotiation successful
0925 22:06:04.677919 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:06:04.678180 15314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:04.675882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:36971 (local address 127.5.0.67:54349)
0925 22:06:04.676122 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:04.676152 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0925 22:06:04.676568 (+ 416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:06:04.676763 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:06:04.676766 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:06:04.676774 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:06:04.677625 (+ 851us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:06:04.677625 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:06:04.677652 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:06:04.677686 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:06:04.677860 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:06:04.677862 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:06:04.677864 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 22:06:04.677913 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 22:06:04.677958 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":97,"threads_started":1}
I20260925 22:06:04.678614 15312 heartbeater.cc:347] Connected to a master server at 127.5.0.126:36971
I20260925 22:06:04.678717 15312 heartbeater.cc:464] Registering TS with master...
I20260925 22:06:04.678854 15312 heartbeater.cc:511] Master 127.5.0.126:36971 requested a full tablet report, sending...
I20260925 22:06:04.679224 14862 ts_manager.cc:194] Registered new tserver with Master: c367411630be4e28be09afbad9b223f3 (127.5.0.67:36343)
I20260925 22:06:04.684317 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 15184
I20260925 22:06:04.684399 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance
I20260925 22:06:04.684542 5121 external_mini_cluster.cc:1428] Setting key d52123480fc8a42d68258aad8856b487
I20260925 22:06:04.685395 5121 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 22:06:04.687337 14920 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:06:04.686801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42632 (local address 127.5.0.126:36971)
0925 22:06:04.686866 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:04.686868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:04.687017 (+ 149us) server_negotiation.cc:407] Connection header received
0925 22:06:04.687060 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:04.687060 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:04.687065 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:04.687128 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:06:04.687271 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42632: BlockingRecv error: recv got EOF from 127.0.0.1:42632 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 22:06:04.687392 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 14922
I20260925 22:06:04.692485 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 15053
I20260925 22:06:04.696997 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 15184
I20260925 22:06:04.701223 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 14831
2026-09-25T22:06:04Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (529 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:06:04 dist-test-slave-7mqr krb5kdc[15322](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:06:04 dist-test-slave-7mqr krb5kdc[15322](info): set up 2 sockets
Sep 25 22:06:04 dist-test-slave-7mqr krb5kdc[15322](info): commencing operation
krb5kdc: starting...
W20260925 22:06:05.748869 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:06:05 dist-test-slave-7mqr krb5kdc[15322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:06:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:06:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.0.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
I20260925 22:06:05.865913 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:45459
--webserver_interface=127.5.0.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:41633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:45459
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:06:05.947320 15335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:06:05.947539 15335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:06:05.947568 15335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:06:05.948776 15335 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:06:05.948818 15335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:06:05.948833 15335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:06:05.948848 15335 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:06:05.948860 15335 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:06:05.950373 15335 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:41633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:45459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:45459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:06:05.950619 15335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:06:05.950862 15335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 22:06:05.955679 15335 server_base.cc:1034] running on GCE node
W20260925 22:06:05.955662 15343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:06:05.955681 15341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:06:05.955662 15340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:06:05.956208 15335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:06:05.956506 15335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:06:05.957676 15335 hybrid_clock.cc:648] HybridClock initialized: now 1790373965957665 us; error 45 us; skew 500 ppm
Sep 25 22:06:05 dist-test-slave-7mqr krb5kdc[15322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373965, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:06:05.960130 15335 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:06:05.960541 15335 webserver.cc:460] Webserver started at http://127.5.0.126:41629/ using document root <none> and password file <none>
I20260925 22:06:05.960786 15335 fs_manager.cc:359] Metadata directory not provided
I20260925 22:06:05.960856 15335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:06:05.961004 15335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:06:05.962035 15335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "42d9023a9be946b19f96a4ccf7231602"
format_stamp: "Formatted at 2026-09-25 22:06:05 on dist-test-slave-7mqr"
server_key: "2a9d8e7eac93ce1c77b853892fce2c9a"
server_key_iv: "2aa5b5ef73ad93d24798b870e2f112a6"
server_key_version: "encryptionkey@0"
I20260925 22:06:05.962391 15335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "42d9023a9be946b19f96a4ccf7231602"
format_stamp: "Formatted at 2026-09-25 22:06:05 on dist-test-slave-7mqr"
server_key: "2a9d8e7eac93ce1c77b853892fce2c9a"
server_key_iv: "2aa5b5ef73ad93d24798b870e2f112a6"
server_key_version: "encryptionkey@0"
I20260925 22:06:05.963701 15335 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260925 22:06:05.964514 15350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:05.964692 15335 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 22:06:05.964790 15335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "42d9023a9be946b19f96a4ccf7231602"
format_stamp: "Formatted at 2026-09-25 22:06:05 on dist-test-slave-7mqr"
server_key: "2a9d8e7eac93ce1c77b853892fce2c9a"
server_key_iv: "2aa5b5ef73ad93d24798b870e2f112a6"
server_key_version: "encryptionkey@0"
I20260925 22:06:05.964872 15335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:06:05.994836 15335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:06:05.996887 15335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:06:05.997058 15335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:06:06.000938 15335 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:45459
I20260925 22:06:06.000973 15402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:45459 every 8 connection(s)
I20260925 22:06:06.001358 15335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:06:06.002367 15403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:06:06.005252 15403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42d9023a9be946b19f96a4ccf7231602: Bootstrap starting.
I20260925 22:06:06.005859 15403 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42d9023a9be946b19f96a4ccf7231602: Neither blocks nor log segments found. Creating new log.
I20260925 22:06:06.006167 15403 log.cc:824] T 00000000000000000000000000000000 P 42d9023a9be946b19f96a4ccf7231602: Log is configured to *not* fsync() on all Append() calls
I20260925 22:06:06.006868 15403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42d9023a9be946b19f96a4ccf7231602: No bootstrap required, opened a new log
I20260925 22:06:06.008459 15403 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42d9023a9be946b19f96a4ccf7231602 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d9023a9be946b19f96a4ccf7231602" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 45459 } }
I20260925 22:06:06.008630 15403 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42d9023a9be946b19f96a4ccf7231602 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:06:06.008654 15403 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42d9023a9be946b19f96a4ccf7231602 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42d9023a9be946b19f96a4ccf7231602, State: Initialized, Role: FOLLOWER
I20260925 22:06:06.008834 15403 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42d9023a9be946b19f96a4ccf7231602 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d9023a9be946b19f96a4ccf7231602" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 45459 } }
I20260925 22:06:06.008903 15403 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42d9023a9be946b19f96a4ccf7231602 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:06:06.008924 15403 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42d9023a9be946b19f96a4ccf7231602 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:06:06.008992 15403 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42d9023a9be946b19f96a4ccf7231602 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:06:06.009766 15403 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42d9023a9be946b19f96a4ccf7231602 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d9023a9be946b19f96a4ccf7231602" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 45459 } }
I20260925 22:06:06.009876 15403 leader_election.cc:302] T 00000000000000000000000000000000 P 42d9023a9be946b19f96a4ccf7231602 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42d9023a9be946b19f96a4ccf7231602; no voters:
I20260925 22:06:06.010041 15403 leader_election.cc:288] T 00000000000000000000000000000000 P 42d9023a9be946b19f96a4ccf7231602 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:06:06.010089 15406 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42d9023a9be946b19f96a4ccf7231602 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:06:06.010246 15406 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42d9023a9be946b19f96a4ccf7231602 [term 1 LEADER]: Becoming Leader. State: Replica: 42d9023a9be946b19f96a4ccf7231602, State: Running, Role: LEADER
I20260925 22:06:06.010303 15403 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42d9023a9be946b19f96a4ccf7231602 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:06:06.010421 15406 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42d9023a9be946b19f96a4ccf7231602 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d9023a9be946b19f96a4ccf7231602" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 45459 } }
I20260925 22:06:06.010715 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 15335
I20260925 22:06:06.010829 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:06:06.010811 15407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42d9023a9be946b19f96a4ccf7231602 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42d9023a9be946b19f96a4ccf7231602" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d9023a9be946b19f96a4ccf7231602" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 45459 } } }
I20260925 22:06:06.010883 15407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42d9023a9be946b19f96a4ccf7231602 [sys.catalog]: This master's current role is: LEADER
I20260925 22:06:06.011068 5121 external_mini_cluster.cc:1428] Setting key 00b7a45486b9e4365d9279a305e406b0
I20260925 22:06:06.011089 15414 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:06:06.011595 15408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42d9023a9be946b19f96a4ccf7231602 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42d9023a9be946b19f96a4ccf7231602. Latest consensus state: current_term: 1 leader_uuid: "42d9023a9be946b19f96a4ccf7231602" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d9023a9be946b19f96a4ccf7231602" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 45459 } } }
I20260925 22:06:06.011757 15408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42d9023a9be946b19f96a4ccf7231602 [sys.catalog]: This master's current role is: LEADER
I20260925 22:06:06.011767 15414 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:06:06.013387 15414 catalog_manager.cc:1347] Generated new cluster ID: be2d2b91a6ba4cb499da40a5b2f20af5
I20260925 22:06:06.013440 15414 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:06:06.020756 15414 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:06:06.021458 15414 catalog_manager.cc:1504] Loading token signing keys...
Sep 25 22:06:06 dist-test-slave-7mqr krb5kdc[15322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:06:06.027271 15424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:06.011771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57314 (local address 127.5.0.126:45459)
0925 22:06:06.016032 (+ 4261us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:06.016041 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:06.016079 (+ 38us) server_negotiation.cc:407] Connection header received
0925 22:06:06.016199 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:06.016206 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:06.016261 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:06.016419 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:06:06.017130 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.018642 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.019683 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.020337 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.020447 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:06.023328 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:06:06.023332 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:06:06.023333 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:06:06.023344 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:06:06.025268 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:06:06.025670 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:06:06.025673 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:06:06.025673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:06:06.025773 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:06:06.025997 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:06:06.025998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:06:06.025999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:06:06.026288 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:06:06.026417 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0925 22:06:06.026869 (+ 452us) server_negotiation.cc:299] Negotiation successful
0925 22:06:06.027029 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4194,"thread_start_us":78,"threads_started":1}
I20260925 22:06:06.036371 15414 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42d9023a9be946b19f96a4ccf7231602: Generated new TSK 0
I20260925 22:06:06.036583 15414 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.0.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
I20260925 22:06:06.066000 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.65:0
--local_ip_for_outbound_sockets=127.5.0.65
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:45459
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:41633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:06:06.146876 15429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:06:06.147104 15429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:06:06.147140 15429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:06:06.148298 15429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:06:06.148360 15429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:06:06.148403 15429 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.65
I20260925 22:06:06.149817 15429 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:41633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:45459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.65
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:06:06.150070 15429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:06:06.150288 15429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:06:06.155066 15434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:06:06.155085 15437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:06:06.155099 15435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:06:06.155102 15429 server_base.cc:1034] running on GCE node
I20260925 22:06:06.155480 15429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:06:06.155694 15429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:06:06.156865 15429 hybrid_clock.cc:648] HybridClock initialized: now 1790373966156835 us; error 45 us; skew 500 ppm
Sep 25 22:06:06 dist-test-slave-7mqr krb5kdc[15322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373966, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:06:06.159232 15429 init.cc:377] Logged in from keytab as kudu/127.5.0.65@KRBTEST.COM (short username kudu)
I20260925 22:06:06.159631 15429 webserver.cc:460] Webserver started at http://127.5.0.65:34441/ using document root <none> and password file <none>
I20260925 22:06:06.159870 15429 fs_manager.cc:359] Metadata directory not provided
I20260925 22:06:06.159986 15429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:06:06.160140 15429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:06:06.161152 15429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/data/instance:
uuid: "e1559ff68d3d481aaad7143cff786623"
format_stamp: "Formatted at 2026-09-25 22:06:06 on dist-test-slave-7mqr"
server_key: "e3013fbdc16bd1ab6bccf9519414d585"
server_key_iv: "63dd54534b895f55c5cb444a649f445b"
server_key_version: "encryptionkey@0"
I20260925 22:06:06.161505 15429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance:
uuid: "e1559ff68d3d481aaad7143cff786623"
format_stamp: "Formatted at 2026-09-25 22:06:06 on dist-test-slave-7mqr"
server_key: "e3013fbdc16bd1ab6bccf9519414d585"
server_key_iv: "63dd54534b895f55c5cb444a649f445b"
server_key_version: "encryptionkey@0"
I20260925 22:06:06.162763 15429 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:06:06.163422 15444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:06.163587 15429 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:06:06.163671 15429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/wal
uuid: "e1559ff68d3d481aaad7143cff786623"
format_stamp: "Formatted at 2026-09-25 22:06:06 on dist-test-slave-7mqr"
server_key: "e3013fbdc16bd1ab6bccf9519414d585"
server_key_iv: "63dd54534b895f55c5cb444a649f445b"
server_key_version: "encryptionkey@0"
I20260925 22:06:06.163724 15429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:06:06.175313 15429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:06:06.177199 15429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:06:06.177307 15429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:06:06.177480 15429 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:06:06.177749 15429 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:06:06.177781 15429 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:06.177805 15429 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:06:06.177819 15429 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:06.183641 15429 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.65:46881
I20260925 22:06:06.183686 15557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.65:46881 every 8 connection(s)
I20260925 22:06:06.184098 15429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
Sep 25 22:06:06 dist-test-slave-7mqr krb5kdc[15322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373966, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:06:06.191455 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 15429
I20260925 22:06:06.191555 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance
I20260925 22:06:06.191814 5121 external_mini_cluster.cc:1428] Setting key c92b1597eb41fb8141e6d37bbe3effaf
I20260925 22:06:06.193964 15424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:06.184751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:42003 (local address 127.5.0.126:45459)
0925 22:06:06.184797 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:06.184799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:06.185038 (+ 239us) server_negotiation.cc:407] Connection header received
0925 22:06:06.185552 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:06.185554 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:06.185564 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:06.185624 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:06:06.186617 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.187150 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.188211 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.188413 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.188522 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:06.190680 (+ 2158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:06:06.190681 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:06:06.190681 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:06:06.190688 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:06:06.192296 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:06:06.192872 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:06:06.192873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:06:06.192874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:06:06.192939 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:06:06.193210 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:06:06.193211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:06:06.193211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:06:06.193364 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:06:06.193433 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 22:06:06.193711 (+ 278us) server_negotiation.cc:299] Negotiation successful
0925 22:06:06.193810 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:06:06.193997 15560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:06.184753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:45459 (local address 127.5.0.65:42003)
0925 22:06:06.184909 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:06.184943 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 22:06:06.185437 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:06:06.185684 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:06:06.185688 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:06:06.186288 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:06:06.186517 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:06.186523 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.187237 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.187238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:06.188122 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:06.188125 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.188532 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.188533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:06.188642 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:06.189136 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:06:06.189144 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:06:06.190597 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:06:06.192383 (+ 1786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:06:06.192391 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:06:06.192393 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:06:06.192775 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:06:06.193036 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:06:06.193036 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:06:06.193037 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:06:06.193141 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:06:06.193428 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:06:06.193432 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:06:06.193578 (+ 146us) client_negotiation.cc:769] Sending connection context
0925 22:06:06.193668 (+ 90us) client_negotiation.cc:240] Negotiation successful
0925 22:06:06.193792 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":94,"threads_started":1}
I20260925 22:06:06.194550 15558 heartbeater.cc:347] Connected to a master server at 127.5.0.126:45459
I20260925 22:06:06.194658 15558 heartbeater.cc:464] Registering TS with master...
I20260925 22:06:06.194855 15558 heartbeater.cc:511] Master 127.5.0.126:45459 requested a full tablet report, sending...
I20260925 22:06:06.195393 15367 ts_manager.cc:194] Registered new tserver with Master: e1559ff68d3d481aaad7143cff786623 (127.5.0.65:46881)
I20260925 22:06:06.196142 15367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.65@KRBTEST.COM'} at 127.5.0.65:42003
WARNING: no policy specified for kudu/127.5.0.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
I20260925 22:06:06.216455 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.66:0
--local_ip_for_outbound_sockets=127.5.0.66
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:45459
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:41633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:06:06.297720 15563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:06:06.297947 15563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:06:06.297983 15563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:06:06.299122 15563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:06:06.299201 15563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:06:06.299247 15563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.66
I20260925 22:06:06.300743 15563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:41633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:45459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.66
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:06:06.301003 15563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:06:06.301231 15563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:06:06.306038 15568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:06:06.306092 15569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:06:06.306075 15571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:06:06.306241 15563 server_base.cc:1034] running on GCE node
I20260925 22:06:06.306435 15563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:06:06.306632 15563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:06:06.307770 15563 hybrid_clock.cc:648] HybridClock initialized: now 1790373966307752 us; error 32 us; skew 500 ppm
Sep 25 22:06:06 dist-test-slave-7mqr krb5kdc[15322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373966, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:06:06.310194 15563 init.cc:377] Logged in from keytab as kudu/127.5.0.66@KRBTEST.COM (short username kudu)
I20260925 22:06:06.310762 15563 webserver.cc:460] Webserver started at http://127.5.0.66:43851/ using document root <none> and password file <none>
I20260925 22:06:06.311004 15563 fs_manager.cc:359] Metadata directory not provided
I20260925 22:06:06.311069 15563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:06:06.311206 15563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:06:06.312403 15563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/data/instance:
uuid: "d8f3b26232a94484841ab0206d49d7be"
format_stamp: "Formatted at 2026-09-25 22:06:06 on dist-test-slave-7mqr"
server_key: "987f3f1ffcae81def056f7509b538b68"
server_key_iv: "8273944f82c6b4a0e5e4be618ff8fc52"
server_key_version: "encryptionkey@0"
I20260925 22:06:06.312765 15563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance:
uuid: "d8f3b26232a94484841ab0206d49d7be"
format_stamp: "Formatted at 2026-09-25 22:06:06 on dist-test-slave-7mqr"
server_key: "987f3f1ffcae81def056f7509b538b68"
server_key_iv: "8273944f82c6b4a0e5e4be618ff8fc52"
server_key_version: "encryptionkey@0"
I20260925 22:06:06.314055 15563 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:06:06.314767 15578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:06.314942 15563 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:06:06.315029 15563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/wal
uuid: "d8f3b26232a94484841ab0206d49d7be"
format_stamp: "Formatted at 2026-09-25 22:06:06 on dist-test-slave-7mqr"
server_key: "987f3f1ffcae81def056f7509b538b68"
server_key_iv: "8273944f82c6b4a0e5e4be618ff8fc52"
server_key_version: "encryptionkey@0"
I20260925 22:06:06.315116 15563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:06:06.333477 15563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:06:06.335263 15563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:06:06.335433 15563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:06:06.335659 15563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:06:06.335980 15563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:06:06.336033 15563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:06.336076 15563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:06:06.336102 15563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:06.341815 15563 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.66:43365
I20260925 22:06:06.341862 15691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.66:43365 every 8 connection(s)
I20260925 22:06:06.342212 15563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
Sep 25 22:06:06 dist-test-slave-7mqr krb5kdc[15322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373966, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:06:06.351089 15424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:06.342948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:33973 (local address 127.5.0.126:45459)
0925 22:06:06.343034 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:06.343037 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:06.343216 (+ 179us) server_negotiation.cc:407] Connection header received
0925 22:06:06.343727 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:06.343728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:06.343740 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:06.343798 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:06:06.344754 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.345256 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.346240 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.346459 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.346546 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:06.348570 (+ 2024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:06:06.348572 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:06:06.348572 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:06:06.348580 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:06:06.349761 (+ 1181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:06:06.350149 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:06:06.350150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:06:06.350151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:06:06.350214 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:06:06.350462 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:06:06.350463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:06:06.350463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:06:06.350573 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:06:06.350650 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 22:06:06.350914 (+ 264us) server_negotiation.cc:299] Negotiation successful
0925 22:06:06.350998 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 22:06:06.351271 15694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:06.342916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:45459 (local address 127.5.0.66:33973)
0925 22:06:06.343088 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:06.343122 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 22:06:06.343620 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:06:06.343868 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:06:06.343872 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:06:06.344469 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:06:06.344654 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:06.344659 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.345357 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.345358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:06.346127 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:06.346129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.346554 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.346555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:06.346676 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:06.347078 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:06:06.347086 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:06:06.348476 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:06:06.349854 (+ 1378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:06:06.349859 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:06:06.349861 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:06:06.350074 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:06:06.350284 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:06:06.350284 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:06:06.350285 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:06:06.350386 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:06:06.350640 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:06:06.350645 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:06:06.350779 (+ 134us) client_negotiation.cc:769] Sending connection context
0925 22:06:06.350879 (+ 100us) client_negotiation.cc:240] Negotiation successful
0925 22:06:06.351033 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":83,"threads_started":1}
I20260925 22:06:06.351747 15692 heartbeater.cc:347] Connected to a master server at 127.5.0.126:45459
I20260925 22:06:06.351822 15692 heartbeater.cc:464] Registering TS with master...
I20260925 22:06:06.352002 15692 heartbeater.cc:511] Master 127.5.0.126:45459 requested a full tablet report, sending...
I20260925 22:06:06.352367 15367 ts_manager.cc:194] Registered new tserver with Master: d8f3b26232a94484841ab0206d49d7be (127.5.0.66:43365)
I20260925 22:06:06.352361 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 15563
I20260925 22:06:06.352474 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance
I20260925 22:06:06.352662 5121 external_mini_cluster.cc:1428] Setting key b2551535d684abf4da7cdd7ab179a142
I20260925 22:06:06.352818 15367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.66@KRBTEST.COM'} at 127.5.0.66:33973
WARNING: no policy specified for kudu/127.5.0.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
I20260925 22:06:06.376727 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.67:0
--local_ip_for_outbound_sockets=127.5.0.67
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:45459
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:41633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:06:06.454841 15697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:06:06.455037 15697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:06:06.455060 15697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:06:06.456235 15697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:06:06.456277 15697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:06:06.456303 15697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.67
I20260925 22:06:06.457672 15697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:41633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:45459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.67
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:06:06.457903 15697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:06:06.458148 15697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:06:06.462883 15702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:06:06.462932 15697 server_base.cc:1034] running on GCE node
W20260925 22:06:06.462901 15703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:06:06.462901 15705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:06:06.463282 15697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:06:06.463479 15697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:06:06.464665 15697 hybrid_clock.cc:648] HybridClock initialized: now 1790373966464643 us; error 34 us; skew 500 ppm
Sep 25 22:06:06 dist-test-slave-7mqr krb5kdc[15322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373966, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:06:06.467075 15697 init.cc:377] Logged in from keytab as kudu/127.5.0.67@KRBTEST.COM (short username kudu)
I20260925 22:06:06.467532 15697 webserver.cc:460] Webserver started at http://127.5.0.67:35617/ using document root <none> and password file <none>
I20260925 22:06:06.467751 15697 fs_manager.cc:359] Metadata directory not provided
I20260925 22:06:06.467814 15697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:06:06.467976 15697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:06:06.468993 15697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/data/instance:
uuid: "62c03e7bf5ec437ab192a154490a0632"
format_stamp: "Formatted at 2026-09-25 22:06:06 on dist-test-slave-7mqr"
server_key: "25d9d0900912fdce8931adaa6606baa9"
server_key_iv: "a99d59a57dde1d268a72794677471209"
server_key_version: "encryptionkey@0"
I20260925 22:06:06.469347 15697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance:
uuid: "62c03e7bf5ec437ab192a154490a0632"
format_stamp: "Formatted at 2026-09-25 22:06:06 on dist-test-slave-7mqr"
server_key: "25d9d0900912fdce8931adaa6606baa9"
server_key_iv: "a99d59a57dde1d268a72794677471209"
server_key_version: "encryptionkey@0"
I20260925 22:06:06.470634 15697 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:06:06.471342 15712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:06.471526 15697 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:06:06.471623 15697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/wal
uuid: "62c03e7bf5ec437ab192a154490a0632"
format_stamp: "Formatted at 2026-09-25 22:06:06 on dist-test-slave-7mqr"
server_key: "25d9d0900912fdce8931adaa6606baa9"
server_key_iv: "a99d59a57dde1d268a72794677471209"
server_key_version: "encryptionkey@0"
I20260925 22:06:06.471701 15697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:06:06.482019 15697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:06:06.483769 15697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:06:06.483979 15697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:06:06.484203 15697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:06:06.484503 15697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:06:06.484562 15697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:06.484602 15697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:06:06.484635 15697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:06.490253 15697 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.67:37225
I20260925 22:06:06.490324 15825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.67:37225 every 8 connection(s)
I20260925 22:06:06.490634 15697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
I20260925 22:06:06.491158 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 15697
I20260925 22:06:06.491245 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance
I20260925 22:06:06.491477 5121 external_mini_cluster.cc:1428] Setting key 0ff3faba2338d7e4a31b87804c2c9083
Sep 25 22:06:06 dist-test-slave-7mqr krb5kdc[15322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373966, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:06:06.499286 15424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:06.491367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:37679 (local address 127.5.0.126:45459)
0925 22:06:06.491443 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:06.491446 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:06.491650 (+ 204us) server_negotiation.cc:407] Connection header received
0925 22:06:06.492376 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:06.492377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:06.492386 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:06.492431 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:06:06.493304 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.493797 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.494715 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.494886 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.494935 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:06.497048 (+ 2113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:06:06.497049 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:06:06.497049 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:06:06.497055 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:06:06.498234 (+ 1179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:06:06.498567 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:06:06.498567 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:06:06.498568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:06:06.498614 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:06:06.498813 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:06:06.498813 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:06:06.498813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:06:06.498907 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:06:06.498956 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 22:06:06.499166 (+ 210us) server_negotiation.cc:299] Negotiation successful
0925 22:06:06.499224 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 22:06:06.499508 15828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:06.491342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:45459 (local address 127.5.0.67:37679)
0925 22:06:06.491498 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:06.491548 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 22:06:06.492273 (+ 725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:06:06.492483 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:06:06.492486 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:06:06.493027 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:06:06.493219 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:06.493223 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.493874 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.493875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:06.494640 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:06.494642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.494952 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.494952 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:06.495043 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:06.495564 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:06:06.495580 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:06:06.496965 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:06:06.498302 (+ 1337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:06:06.498307 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:06:06.498308 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:06:06.498506 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:06:06.498672 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:06:06.498673 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:06:06.498673 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:06:06.498762 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:06:06.498959 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:06:06.498963 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:06:06.499065 (+ 102us) client_negotiation.cc:769] Sending connection context
0925 22:06:06.499141 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 22:06:06.499263 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":83,"threads_started":1}
I20260925 22:06:06.500020 15826 heartbeater.cc:347] Connected to a master server at 127.5.0.126:45459
I20260925 22:06:06.500119 15826 heartbeater.cc:464] Registering TS with master...
I20260925 22:06:06.500303 15826 heartbeater.cc:511] Master 127.5.0.126:45459 requested a full tablet report, sending...
I20260925 22:06:06.500654 15367 ts_manager.cc:194] Registered new tserver with Master: 62c03e7bf5ec437ab192a154490a0632 (127.5.0.67:37225)
I20260925 22:06:06.501030 15367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.67@KRBTEST.COM'} at 127.5.0.67:37679
I20260925 22:06:06.502946 5121 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:06:06.510135 15424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:06.504227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57326 (local address 127.5.0.126:45459)
0925 22:06:06.504306 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:06.504308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:06.504404 (+ 96us) server_negotiation.cc:407] Connection header received
0925 22:06:06.504476 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:06.504477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:06.504486 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:06.504542 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:06:06.505082 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.505546 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.506266 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.506480 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.506564 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:06.507722 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:06:06.507724 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:06:06.507724 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:06:06.507731 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:06:06.508956 (+ 1225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:06:06.509300 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:06:06.509301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:06:06.509302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:06:06.509354 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:06:06.509553 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:06:06.509554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:06:06.509554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:06:06.509751 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:06:06.509835 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 22:06:06.509966 (+ 131us) server_negotiation.cc:299] Negotiation successful
0925 22:06:06.510039 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 22:06:06.511621 15367 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57326:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:06:06.513262 15367 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:06:06.519518 15834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:06.515198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:43365 (local address 127.0.0.1:50348)
0925 22:06:06.515315 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:06.515350 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 22:06:06.515409 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:06:06.515831 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:06:06.515834 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:06:06.515836 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:06:06.515925 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:06.515927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.517234 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.517235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:06.518488 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:06.518491 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.519216 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.519217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:06.519305 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:06.519318 (+ 13us) client_negotiation.cc:769] Sending connection context
0925 22:06:06.519391 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 22:06:06.519444 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":55,"thread_start_us":74,"threads_started":1}
I20260925 22:06:06.519536 15838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:06.515479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50348 (local address 127.5.0.66:43365)
0925 22:06:06.515668 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:06.515672 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:06.515684 (+ 12us) server_negotiation.cc:407] Connection header received
0925 22:06:06.515724 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:06.515726 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:06.515743 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:06.515828 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:06:06.516012 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.517135 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.518587 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.519135 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.519217 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:06.519274 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 22:06:06.519416 (+ 142us) server_negotiation.cc:299] Negotiation successful
0925 22:06:06.519454 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":116,"threads_started":1}
I20260925 22:06:06.519757 15836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:06.515233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:46881 (local address 127.0.0.1:47308)
0925 22:06:06.515682 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:06.515693 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:06:06.515743 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:06:06.516152 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:06:06.516152 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:06:06.516154 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:06:06.516202 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:06.516204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.517414 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.517415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:06.518440 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:06.518442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.519487 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.519488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:06.519560 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:06.519562 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 22:06:06.519635 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 22:06:06.519681 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":51,"threads_started":1}
I20260925 22:06:06.520015 15839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:06.515603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47308 (local address 127.5.0.65:46881)
0925 22:06:06.515777 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:06.515780 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:06.515790 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:06:06.515829 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:06.515831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:06.515846 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:06.516078 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:06:06.516286 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.517316 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.518524 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.519329 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.519399 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:06.519450 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 22:06:06.519850 (+ 400us) server_negotiation.cc:299] Negotiation successful
0925 22:06:06.519898 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":111,"threads_started":1}
I20260925 22:06:06.520047 15837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:06.515431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60602 (local address 127.5.0.67:37225)
0925 22:06:06.515573 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:06.515577 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:06.515874 (+ 297us) server_negotiation.cc:407] Connection header received
0925 22:06:06.516000 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:06.516003 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:06.516025 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:06.516094 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:06:06.516289 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.516947 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.518603 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.519330 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.519399 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:06.519450 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 22:06:06.519872 (+ 422us) server_negotiation.cc:299] Negotiation successful
0925 22:06:06.519939 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":80,"threads_started":1}
I20260925 22:06:06.520223 15835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:06.515233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:37225 (local address 127.0.0.1:60602)
0925 22:06:06.515820 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:06.515828 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 22:06:06.515923 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:06:06.516164 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:06:06.516165 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:06:06.516167 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:06:06.516221 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:06.516223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.517062 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.517063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:06.518488 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:06.518491 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.519716 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.519717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:06.519775 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:06.519779 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:06:06.520118 (+ 339us) client_negotiation.cc:240] Negotiation successful
0925 22:06:06.520161 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"mutex_wait_us":105,"thread_start_us":54,"threads_started":1}
I20260925 22:06:06.520367 15626 tablet_service.cc:1467] Processing CreateTablet for tablet 4043c344b7b040208bece2004d476ee2 (DEFAULT_TABLE table=test-table [id=e9f02a8d8d104d628d26375d099ba3cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:06:06.520694 15626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4043c344b7b040208bece2004d476ee2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:06:06.520715 15492 tablet_service.cc:1467] Processing CreateTablet for tablet 4043c344b7b040208bece2004d476ee2 (DEFAULT_TABLE table=test-table [id=e9f02a8d8d104d628d26375d099ba3cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:06:06.520821 15760 tablet_service.cc:1467] Processing CreateTablet for tablet 4043c344b7b040208bece2004d476ee2 (DEFAULT_TABLE table=test-table [id=e9f02a8d8d104d628d26375d099ba3cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:06:06.521037 15492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4043c344b7b040208bece2004d476ee2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:06:06.521071 15760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4043c344b7b040208bece2004d476ee2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:06:06.523067 15840 tablet_bootstrap.cc:492] T 4043c344b7b040208bece2004d476ee2 P d8f3b26232a94484841ab0206d49d7be: Bootstrap starting.
I20260925 22:06:06.523627 15840 tablet_bootstrap.cc:654] T 4043c344b7b040208bece2004d476ee2 P d8f3b26232a94484841ab0206d49d7be: Neither blocks nor log segments found. Creating new log.
I20260925 22:06:06.524312 15840 log.cc:824] T 4043c344b7b040208bece2004d476ee2 P d8f3b26232a94484841ab0206d49d7be: Log is configured to *not* fsync() on all Append() calls
I20260925 22:06:06.524346 15842 tablet_bootstrap.cc:492] T 4043c344b7b040208bece2004d476ee2 P e1559ff68d3d481aaad7143cff786623: Bootstrap starting.
I20260925 22:06:06.524647 15841 tablet_bootstrap.cc:492] T 4043c344b7b040208bece2004d476ee2 P 62c03e7bf5ec437ab192a154490a0632: Bootstrap starting.
I20260925 22:06:06.525007 15840 tablet_bootstrap.cc:492] T 4043c344b7b040208bece2004d476ee2 P d8f3b26232a94484841ab0206d49d7be: No bootstrap required, opened a new log
I20260925 22:06:06.525035 15842 tablet_bootstrap.cc:654] T 4043c344b7b040208bece2004d476ee2 P e1559ff68d3d481aaad7143cff786623: Neither blocks nor log segments found. Creating new log.
I20260925 22:06:06.525094 15840 ts_tablet_manager.cc:1397] T 4043c344b7b040208bece2004d476ee2 P d8f3b26232a94484841ab0206d49d7be: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 22:06:06.525295 15841 tablet_bootstrap.cc:654] T 4043c344b7b040208bece2004d476ee2 P 62c03e7bf5ec437ab192a154490a0632: Neither blocks nor log segments found. Creating new log.
I20260925 22:06:06.525599 15842 log.cc:824] T 4043c344b7b040208bece2004d476ee2 P e1559ff68d3d481aaad7143cff786623: Log is configured to *not* fsync() on all Append() calls
I20260925 22:06:06.525813 15841 log.cc:824] T 4043c344b7b040208bece2004d476ee2 P 62c03e7bf5ec437ab192a154490a0632: Log is configured to *not* fsync() on all Append() calls
I20260925 22:06:06.526368 15842 tablet_bootstrap.cc:492] T 4043c344b7b040208bece2004d476ee2 P e1559ff68d3d481aaad7143cff786623: No bootstrap required, opened a new log
I20260925 22:06:06.526463 15842 ts_tablet_manager.cc:1397] T 4043c344b7b040208bece2004d476ee2 P e1559ff68d3d481aaad7143cff786623: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:06:06.526583 15841 tablet_bootstrap.cc:492] T 4043c344b7b040208bece2004d476ee2 P 62c03e7bf5ec437ab192a154490a0632: No bootstrap required, opened a new log
I20260925 22:06:06.526651 15841 ts_tablet_manager.cc:1397] T 4043c344b7b040208bece2004d476ee2 P 62c03e7bf5ec437ab192a154490a0632: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:06:06.527062 15840 raft_consensus.cc:348] T 4043c344b7b040208bece2004d476ee2 P d8f3b26232a94484841ab0206d49d7be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8f3b26232a94484841ab0206d49d7be" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 43365 } } peers { permanent_uuid: "62c03e7bf5ec437ab192a154490a0632" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37225 } } peers { permanent_uuid: "e1559ff68d3d481aaad7143cff786623" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46881 } }
I20260925 22:06:06.527225 15840 raft_consensus.cc:374] T 4043c344b7b040208bece2004d476ee2 P d8f3b26232a94484841ab0206d49d7be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:06:06.527258 15840 raft_consensus.cc:729] T 4043c344b7b040208bece2004d476ee2 P d8f3b26232a94484841ab0206d49d7be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8f3b26232a94484841ab0206d49d7be, State: Initialized, Role: FOLLOWER
I20260925 22:06:06.527395 15840 consensus_queue.cc:260] T 4043c344b7b040208bece2004d476ee2 P d8f3b26232a94484841ab0206d49d7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8f3b26232a94484841ab0206d49d7be" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 43365 } } peers { permanent_uuid: "62c03e7bf5ec437ab192a154490a0632" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37225 } } peers { permanent_uuid: "e1559ff68d3d481aaad7143cff786623" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46881 } }
I20260925 22:06:06.527647 15840 ts_tablet_manager.cc:1428] T 4043c344b7b040208bece2004d476ee2 P d8f3b26232a94484841ab0206d49d7be: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260925 22:06:06.528328 15842 raft_consensus.cc:348] T 4043c344b7b040208bece2004d476ee2 P e1559ff68d3d481aaad7143cff786623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8f3b26232a94484841ab0206d49d7be" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 43365 } } peers { permanent_uuid: "62c03e7bf5ec437ab192a154490a0632" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37225 } } peers { permanent_uuid: "e1559ff68d3d481aaad7143cff786623" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46881 } }
I20260925 22:06:06.528472 15842 raft_consensus.cc:374] T 4043c344b7b040208bece2004d476ee2 P e1559ff68d3d481aaad7143cff786623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:06:06.528520 15842 raft_consensus.cc:729] T 4043c344b7b040208bece2004d476ee2 P e1559ff68d3d481aaad7143cff786623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1559ff68d3d481aaad7143cff786623, State: Initialized, Role: FOLLOWER
I20260925 22:06:06.528442 15841 raft_consensus.cc:348] T 4043c344b7b040208bece2004d476ee2 P 62c03e7bf5ec437ab192a154490a0632 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8f3b26232a94484841ab0206d49d7be" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 43365 } } peers { permanent_uuid: "62c03e7bf5ec437ab192a154490a0632" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37225 } } peers { permanent_uuid: "e1559ff68d3d481aaad7143cff786623" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46881 } }
I20260925 22:06:06.528596 15841 raft_consensus.cc:374] T 4043c344b7b040208bece2004d476ee2 P 62c03e7bf5ec437ab192a154490a0632 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:06:06.528640 15841 raft_consensus.cc:729] T 4043c344b7b040208bece2004d476ee2 P 62c03e7bf5ec437ab192a154490a0632 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62c03e7bf5ec437ab192a154490a0632, State: Initialized, Role: FOLLOWER
I20260925 22:06:06.528622 15842 consensus_queue.cc:260] T 4043c344b7b040208bece2004d476ee2 P e1559ff68d3d481aaad7143cff786623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8f3b26232a94484841ab0206d49d7be" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 43365 } } peers { permanent_uuid: "62c03e7bf5ec437ab192a154490a0632" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37225 } } peers { permanent_uuid: "e1559ff68d3d481aaad7143cff786623" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46881 } }
I20260925 22:06:06.528750 15841 consensus_queue.cc:260] T 4043c344b7b040208bece2004d476ee2 P 62c03e7bf5ec437ab192a154490a0632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8f3b26232a94484841ab0206d49d7be" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 43365 } } peers { permanent_uuid: "62c03e7bf5ec437ab192a154490a0632" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37225 } } peers { permanent_uuid: "e1559ff68d3d481aaad7143cff786623" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46881 } }
I20260925 22:06:06.528873 15842 ts_tablet_manager.cc:1428] T 4043c344b7b040208bece2004d476ee2 P e1559ff68d3d481aaad7143cff786623: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 22:06:06.528990 15692 heartbeater.cc:503] Master 127.5.0.126:45459 was elected leader, sending a full tablet report...
I20260925 22:06:06.529019 15826 heartbeater.cc:503] Master 127.5.0.126:45459 was elected leader, sending a full tablet report...
I20260925 22:06:06.528887 15558 heartbeater.cc:503] Master 127.5.0.126:45459 was elected leader, sending a full tablet report...
I20260925 22:06:06.529243 15841 ts_tablet_manager.cc:1428] T 4043c344b7b040208bece2004d476ee2 P 62c03e7bf5ec437ab192a154490a0632: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260925 22:06:06.592760 15693 tablet.cc:2369] T 4043c344b7b040208bece2004d476ee2 P d8f3b26232a94484841ab0206d49d7be: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:06:06.606760 15847 raft_consensus.cc:482] T 4043c344b7b040208bece2004d476ee2 P e1559ff68d3d481aaad7143cff786623 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:06:06.606880 15847 raft_consensus.cc:504] T 4043c344b7b040208bece2004d476ee2 P e1559ff68d3d481aaad7143cff786623 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8f3b26232a94484841ab0206d49d7be" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 43365 } } peers { permanent_uuid: "62c03e7bf5ec437ab192a154490a0632" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37225 } } peers { permanent_uuid: "e1559ff68d3d481aaad7143cff786623" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46881 } }
I20260925 22:06:06.607415 15847 leader_election.cc:288] T 4043c344b7b040208bece2004d476ee2 P e1559ff68d3d481aaad7143cff786623 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8f3b26232a94484841ab0206d49d7be (127.5.0.66:43365), 62c03e7bf5ec437ab192a154490a0632 (127.5.0.67:37225)
I20260925 22:06:06.610539 15560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:06.607586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:37225 (local address 127.5.0.65:60033)
0925 22:06:06.607627 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:06.607641 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 22:06:06.607683 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:06:06.607918 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:06:06.607920 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:06:06.607922 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:06:06.607993 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:06.607996 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.608761 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.608762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:06.609549 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:06.609551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.610285 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.610286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:06.610369 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:06.610373 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:06:06.610426 (+ 53us) client_negotiation.cc:240] Negotiation successful
0925 22:06:06.610471 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260925 22:06:06.610548 15837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:06.607717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:60033 (local address 127.5.0.67:37225)
0925 22:06:06.607762 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:06.607764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:06.607779 (+ 15us) server_negotiation.cc:407] Connection header received
0925 22:06:06.607803 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:06.607804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:06.607813 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:06.607888 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:06:06.608082 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.608661 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.609645 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.610193 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.610259 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:06.610269 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 22:06:06.610432 (+ 163us) server_negotiation.cc:299] Negotiation successful
0925 22:06:06.610470 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 22:06:06.610738 15849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:06.607593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:43365 (local address 127.5.0.65:47991)
0925 22:06:06.607928 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:06.607937 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:06:06.607998 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:06:06.608162 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:06:06.608162 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:06:06.608164 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:06:06.608228 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:06.608230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.608895 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.608896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:06.609726 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:06.609729 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.610426 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.610427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:06.610506 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:06.610537 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 22:06:06.610600 (+ 63us) client_negotiation.cc:240] Negotiation successful
0925 22:06:06.610642 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":142,"thread_start_us":121,"threads_started":1}
I20260925 22:06:06.610735 15838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:06.607653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:47991 (local address 127.5.0.66:43365)
0925 22:06:06.607694 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:06.607696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:06.607991 (+ 295us) server_negotiation.cc:407] Connection header received
0925 22:06:06.608079 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:06.608080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:06.608091 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:06.608141 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:06:06.608296 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.608803 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.609814 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.610343 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.610414 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:06.610426 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0925 22:06:06.610620 (+ 194us) server_negotiation.cc:299] Negotiation successful
0925 22:06:06.610665 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 22:06:06.611055 15646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4043c344b7b040208bece2004d476ee2" candidate_uuid: "e1559ff68d3d481aaad7143cff786623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8f3b26232a94484841ab0206d49d7be" is_pre_election: true
I20260925 22:06:06.611233 15646 raft_consensus.cc:2454] T 4043c344b7b040208bece2004d476ee2 P d8f3b26232a94484841ab0206d49d7be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1559ff68d3d481aaad7143cff786623 in term 0.
I20260925 22:06:06.611192 15780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4043c344b7b040208bece2004d476ee2" candidate_uuid: "e1559ff68d3d481aaad7143cff786623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62c03e7bf5ec437ab192a154490a0632" is_pre_election: true
I20260925 22:06:06.611339 15780 raft_consensus.cc:2454] T 4043c344b7b040208bece2004d476ee2 P 62c03e7bf5ec437ab192a154490a0632 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1559ff68d3d481aaad7143cff786623 in term 0.
I20260925 22:06:06.611459 15447 leader_election.cc:302] T 4043c344b7b040208bece2004d476ee2 P e1559ff68d3d481aaad7143cff786623 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8f3b26232a94484841ab0206d49d7be, e1559ff68d3d481aaad7143cff786623; no voters:
I20260925 22:06:06.611588 15847 raft_consensus.cc:2781] T 4043c344b7b040208bece2004d476ee2 P e1559ff68d3d481aaad7143cff786623 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:06:06.611641 15847 raft_consensus.cc:482] T 4043c344b7b040208bece2004d476ee2 P e1559ff68d3d481aaad7143cff786623 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:06:06.611665 15847 raft_consensus.cc:3037] T 4043c344b7b040208bece2004d476ee2 P e1559ff68d3d481aaad7143cff786623 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:06:06.612411 15847 raft_consensus.cc:504] T 4043c344b7b040208bece2004d476ee2 P e1559ff68d3d481aaad7143cff786623 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8f3b26232a94484841ab0206d49d7be" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 43365 } } peers { permanent_uuid: "62c03e7bf5ec437ab192a154490a0632" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37225 } } peers { permanent_uuid: "e1559ff68d3d481aaad7143cff786623" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46881 } }
I20260925 22:06:06.612568 15847 leader_election.cc:288] T 4043c344b7b040208bece2004d476ee2 P e1559ff68d3d481aaad7143cff786623 [CANDIDATE]: Term 1 election: Requested vote from peers d8f3b26232a94484841ab0206d49d7be (127.5.0.66:43365), 62c03e7bf5ec437ab192a154490a0632 (127.5.0.67:37225)
I20260925 22:06:06.612691 15646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4043c344b7b040208bece2004d476ee2" candidate_uuid: "e1559ff68d3d481aaad7143cff786623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8f3b26232a94484841ab0206d49d7be"
I20260925 22:06:06.612747 15780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4043c344b7b040208bece2004d476ee2" candidate_uuid: "e1559ff68d3d481aaad7143cff786623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62c03e7bf5ec437ab192a154490a0632"
I20260925 22:06:06.612773 15646 raft_consensus.cc:3037] T 4043c344b7b040208bece2004d476ee2 P d8f3b26232a94484841ab0206d49d7be [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:06:06.612809 15780 raft_consensus.cc:3037] T 4043c344b7b040208bece2004d476ee2 P 62c03e7bf5ec437ab192a154490a0632 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:06:06.613648 15646 raft_consensus.cc:2454] T 4043c344b7b040208bece2004d476ee2 P d8f3b26232a94484841ab0206d49d7be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1559ff68d3d481aaad7143cff786623 in term 1.
I20260925 22:06:06.613648 15780 raft_consensus.cc:2454] T 4043c344b7b040208bece2004d476ee2 P 62c03e7bf5ec437ab192a154490a0632 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1559ff68d3d481aaad7143cff786623 in term 1.
I20260925 22:06:06.613812 15447 leader_election.cc:302] T 4043c344b7b040208bece2004d476ee2 P e1559ff68d3d481aaad7143cff786623 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8f3b26232a94484841ab0206d49d7be, e1559ff68d3d481aaad7143cff786623; no voters:
I20260925 22:06:06.613931 15847 raft_consensus.cc:2781] T 4043c344b7b040208bece2004d476ee2 P e1559ff68d3d481aaad7143cff786623 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:06:06.614084 15847 raft_consensus.cc:686] T 4043c344b7b040208bece2004d476ee2 P e1559ff68d3d481aaad7143cff786623 [term 1 LEADER]: Becoming Leader. State: Replica: e1559ff68d3d481aaad7143cff786623, State: Running, Role: LEADER
I20260925 22:06:06.614199 15847 consensus_queue.cc:237] T 4043c344b7b040208bece2004d476ee2 P e1559ff68d3d481aaad7143cff786623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8f3b26232a94484841ab0206d49d7be" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 43365 } } peers { permanent_uuid: "62c03e7bf5ec437ab192a154490a0632" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37225 } } peers { permanent_uuid: "e1559ff68d3d481aaad7143cff786623" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46881 } }
I20260925 22:06:06.614987 15366 catalog_manager.cc:5495] T 4043c344b7b040208bece2004d476ee2 P e1559ff68d3d481aaad7143cff786623 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1559ff68d3d481aaad7143cff786623 (127.5.0.65). New cstate: current_term: 1 leader_uuid: "e1559ff68d3d481aaad7143cff786623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8f3b26232a94484841ab0206d49d7be" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 43365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62c03e7bf5ec437ab192a154490a0632" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1559ff68d3d481aaad7143cff786623" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 46881 } health_report { overall_health: HEALTHY } } }
I20260925 22:06:06.635288 15839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:06.632875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47322 (local address 127.5.0.65:46881)
0925 22:06:06.632948 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:06.632950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:06.632957 (+ 7us) server_negotiation.cc:407] Connection header received
0925 22:06:06.632996 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:06.632996 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:06.633013 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:06.633090 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:06:06.633240 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.633718 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.634476 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:06.634693 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:06.634768 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:06.634942 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:06:06.635046 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:06:06.635140 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0925 22:06:06.635164 (+ 24us) server_negotiation.cc:299] Negotiation successful
0925 22:06:06.635183 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 22:06:06.637240 15646 raft_consensus.cc:1261] T 4043c344b7b040208bece2004d476ee2 P d8f3b26232a94484841ab0206d49d7be [term 1 FOLLOWER]: Refusing update from remote peer e1559ff68d3d481aaad7143cff786623: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:06:06.637244 15780 raft_consensus.cc:1261] T 4043c344b7b040208bece2004d476ee2 P 62c03e7bf5ec437ab192a154490a0632 [term 1 FOLLOWER]: Refusing update from remote peer e1559ff68d3d481aaad7143cff786623: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:06:06.637559 15847 consensus_queue.cc:1035] T 4043c344b7b040208bece2004d476ee2 P e1559ff68d3d481aaad7143cff786623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62c03e7bf5ec437ab192a154490a0632" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 37225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:06:06.637687 15850 consensus_queue.cc:1035] T 4043c344b7b040208bece2004d476ee2 P e1559ff68d3d481aaad7143cff786623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8f3b26232a94484841ab0206d49d7be" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 43365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:06:06.641086 15854 mvcc.cc:205] Tried to move back new op lower bound from 7333371767343198208 to 7333371767252242432. Current Snapshot: MvccSnapshot[applied={T|T < 7333371767343198208}]
I20260925 22:06:06.642386 15366 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:06:06.642463 15366 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:06:06.643188 15366 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 42d9023a9be946b19f96a4ccf7231602: Sending DeleteTablet for 3 replicas of tablet 4043c344b7b040208bece2004d476ee2
I20260925 22:06:06.643501 15492 tablet_service.cc:1514] Processing DeleteTablet for tablet 4043c344b7b040208bece2004d476ee2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:06:06 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:47308
I20260925 22:06:06.643496 15626 tablet_service.cc:1514] Processing DeleteTablet for tablet 4043c344b7b040208bece2004d476ee2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:06:06 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:50348
I20260925 22:06:06.643502 15760 tablet_service.cc:1514] Processing DeleteTablet for tablet 4043c344b7b040208bece2004d476ee2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:06:06 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:60602
I20260925 22:06:06.643805 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 15429
I20260925 22:06:06.644564 15863 tablet_replica.cc:330] T 4043c344b7b040208bece2004d476ee2 P d8f3b26232a94484841ab0206d49d7be: stopping tablet replica
I20260925 22:06:06.644673 15863 raft_consensus.cc:2229] T 4043c344b7b040208bece2004d476ee2 P d8f3b26232a94484841ab0206d49d7be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:06:06.644680 15864 tablet_replica.cc:330] T 4043c344b7b040208bece2004d476ee2 P 62c03e7bf5ec437ab192a154490a0632: stopping tablet replica
I20260925 22:06:06.644785 15863 raft_consensus.cc:2258] T 4043c344b7b040208bece2004d476ee2 P d8f3b26232a94484841ab0206d49d7be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:06:06.644898 15864 raft_consensus.cc:2229] T 4043c344b7b040208bece2004d476ee2 P 62c03e7bf5ec437ab192a154490a0632 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:06:06.645177 15864 raft_consensus.cc:2258] T 4043c344b7b040208bece2004d476ee2 P 62c03e7bf5ec437ab192a154490a0632 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:06:06.645180 15863 ts_tablet_manager.cc:1905] T 4043c344b7b040208bece2004d476ee2 P d8f3b26232a94484841ab0206d49d7be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:06:06.645582 15864 ts_tablet_manager.cc:1905] T 4043c344b7b040208bece2004d476ee2 P 62c03e7bf5ec437ab192a154490a0632: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:06:06.646934 15863 ts_tablet_manager.cc:1918] T 4043c344b7b040208bece2004d476ee2 P d8f3b26232a94484841ab0206d49d7be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:06:06.647011 15863 log.cc:1196] T 4043c344b7b040208bece2004d476ee2 P d8f3b26232a94484841ab0206d49d7be: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-1/wal/wals/4043c344b7b040208bece2004d476ee2
I20260925 22:06:06.647271 15864 ts_tablet_manager.cc:1918] T 4043c344b7b040208bece2004d476ee2 P 62c03e7bf5ec437ab192a154490a0632: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:06:06.647300 15863 ts_tablet_manager.cc:1939] T 4043c344b7b040208bece2004d476ee2 P d8f3b26232a94484841ab0206d49d7be: Deleting consensus metadata
I20260925 22:06:06.647343 15864 log.cc:1196] T 4043c344b7b040208bece2004d476ee2 P 62c03e7bf5ec437ab192a154490a0632: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373908987421-5121-0/minicluster-data/ts-2/wal/wals/4043c344b7b040208bece2004d476ee2
I20260925 22:06:06.647615 15864 ts_tablet_manager.cc:1939] T 4043c344b7b040208bece2004d476ee2 P 62c03e7bf5ec437ab192a154490a0632: Deleting consensus metadata
I20260925 22:06:06.647723 15353 catalog_manager.cc:4841] TS d8f3b26232a94484841ab0206d49d7be (127.5.0.66:43365): tablet 4043c344b7b040208bece2004d476ee2 (table test-table [id=e9f02a8d8d104d628d26375d099ba3cc]) successfully deleted
I20260925 22:06:06.647998 15351 catalog_manager.cc:4841] TS 62c03e7bf5ec437ab192a154490a0632 (127.5.0.67:37225): tablet 4043c344b7b040208bece2004d476ee2 (table test-table [id=e9f02a8d8d104d628d26375d099ba3cc]) successfully deleted
W20260925 22:06:06.649151 15354 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.0.65:46881 (error 108)
W20260925 22:06:06.649617 15835 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:06:06.649500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:46881 (local address 127.0.0.1:47336)
0925 22:06:06.649555 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:06.649589 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.65:46881: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":38}
W20260925 22:06:06.649725 15354 catalog_manager.cc:4584] TS e1559ff68d3d481aaad7143cff786623 (127.5.0.65:46881): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4043c344b7b040208bece2004d476ee2: Network error: Client connection negotiation failed: client connection to 127.5.0.65:46881: connect: Connection reset by peer (error 104)
I20260925 22:06:06.649784 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 15563
I20260925 22:06:06.654980 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 15697
I20260925 22:06:06.659160 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 15335
2026-09-25T22:06:06Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1958 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:06:06 dist-test-slave-7mqr krb5kdc[15869](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:06:06 dist-test-slave-7mqr krb5kdc[15869](info): set up 2 sockets
Sep 25 22:06:06 dist-test-slave-7mqr krb5kdc[15869](info): commencing operation
krb5kdc: starting...
W20260925 22:06:07.689548 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:06:07 dist-test-slave-7mqr krb5kdc[15869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:06:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:06:07Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.5.0.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.5.0.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.5.0.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/rangeradmin_127.5.0.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.5.0.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/rangeradmin_127.5.0.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.5.0.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.5.0.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.5.0.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/rangerlookup_127.5.0.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.5.0.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/rangerlookup_127.5.0.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.0.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.5.0.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.0.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/HTTP_127.5.0.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.0.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/HTTP_127.5.0.84@KRBTEST.COM.keytab.
I20260925 22:06:07.852857 5121 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskMOkht1/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/postgres -l logfile start
2026-09-25 22:06:10.261 UTC [15899] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-25 22:06:10.261 UTC [15899] LOG: listening on IPv4 address "127.5.0.84", port 45883
2026-09-25 22:06:10.266 UTC [15899] LOG: listening on Unix socket "/tmp/.s.PGSQL.45883"
2026-09-25 22:06:10.287 UTC [15902] LOG: database system was shut down at 2026-09-25 22:06:08 UTC
2026-09-25 22:06:10.291 UTC [15899] LOG: database system is ready to accept connections
I20260925 22:06:11.277446 5121 mini_postgres.cc:96] Postgres bound to 45883
2026-09-25 22:06:11.283 UTC [15910] FATAL: database "slave" does not exist
127.5.0.84:45883 - accepting connections
I20260925 22:06:11.284497 5121 mini_ranger.cc:161] Starting Ranger...
I20260925 22:06:11.299070 5121 mini_ranger.cc:84] Created miniranger Postgres user
I20260925 22:06:11.751866 5121 mini_ranger.cc:87] Created ranger Postgres database
I20260925 22:06:11.751971 5121 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ranger-admin
2026-09-25 22:06:12,021 [I] DB FLAVOR :POSTGRES
2026-09-25 22:06:12,022 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 22:06:12,022 [I] Checking connection..
2026-09-25 22:06:12,022 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:45883/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-25 22:06:12,255 [I] Checking connection passed.
2026-09-25 22:06:12,256 [I] --------- Verifying version history table ---------
2026-09-25 22:06:12,256 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:45883/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 22:06:12,510 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 22:06:12,511 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:45883/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 22:06:12,738 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 22:06:12,739 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-25 22:06:12,739 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:45883/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-25 22:06:12.959 UTC [15974] WARNING: there is no transaction in progress
2026-09-25 22:06:12,965 [I] create_dbversion_catalog.sql file imported successfully
2026-09-25 22:06:12,965 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:45883/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 22:06:13,207 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-25 22:06:13,207 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 22:06:13,207 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:45883/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-25 22:06:13,460 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:45883/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-25 22:06:13,680 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:45883/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-7mqr.c.gcp-upstream.internal','N') ;"
2026-09-25 22:06:13,899 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:45883/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 22:06:14,155 [I] Table x_portal_user does not exist in database ranger
2026-09-25 22:06:14,155 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:45883/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 22:06:14,377 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-25 22:06:14,377 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:45883/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 22:06:14,626 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:45883/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 22:06:14,891 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-25 22:06:14,891 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:45883/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-25 22:06:15.765 UTC [16082] WARNING: there is no transaction in progress
2026-09-25 22:06:15.780 UTC [16082] WARNING: there is no transaction in progress
2026-09-25 22:06:15.796 UTC [16082] WARNING: there is no transaction in progress
2026-09-25 22:06:15.811 UTC [16082] WARNING: there is no transaction in progress
2026-09-25 22:06:15.826 UTC [16082] WARNING: there is no transaction in progress
2026-09-25 22:06:15.843 UTC [16082] WARNING: there is no transaction in progress
2026-09-25 22:06:15.896 UTC [16082] WARNING: there is no transaction in progress
2026-09-25 22:06:15.902 UTC [16082] WARNING: there is no transaction in progress
2026-09-25 22:06:15.916 UTC [16082] WARNING: there is no transaction in progress
2026-09-25 22:06:15.926 UTC [16082] WARNING: there is no transaction in progress
2026-09-25 22:06:15.936 UTC [16082] WARNING: there is no transaction in progress
2026-09-25 22:06:15.949 UTC [16082] WARNING: there is no transaction in progress
2026-09-25 22:06:15.958 UTC [16082] WARNING: there is no transaction in progress
2026-09-25 22:06:15.964 UTC [16082] WARNING: there is no transaction in progress
2026-09-25 22:06:15.970 UTC [16082] WARNING: there is no transaction in progress
2026-09-25 22:06:15.976 UTC [16082] WARNING: there is no transaction in progress
2026-09-25 22:06:16,584 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-25 22:06:16,584 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:45883/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-25 22:06:16,809 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-25 22:06:16,809 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:45883/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 22:06:17,045 [I] Table x_portal_user already exists in database 'ranger'
2026-09-25 22:06:17,045 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:45883/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 22:06:17,279 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-25 22:06:17,279 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:45883/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 22:06:17,526 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:45883/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 22:06:17,750 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:45883/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-7mqr.c.gcp-upstream.internal';"
2026-09-25 22:06:17,979 [I] CORE_DB_SCHEMA import status has been updated
2026-09-25 22:06:17,980 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:45883/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-25 22:06:18,222 [I] DB_PATCHES have already been applied
I20260925 22:06:18.228322 5121 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ranger-admin:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskMOkht1/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260925 22:06:18.228390 5121 mini_ranger.cc:193] Using host: 127.5.0.84
I20260925 22:06:18.231439 5121 mini_ranger.cc:227] Ranger admin URL: http://127.5.0.84:45353
Sep 25, 2026 10:06:18 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 10:06:18 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 10:06:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ranger-admin/ews/webapp
Sep 25, 2026 10:06:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 25, 2026 10:06:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 25, 2026 10:06:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ranger-admin/ews/webapp].
Sep 25, 2026 10:06:19 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260925 22:06:19.359572 5121 mini_ranger.cc:160] Time spent starting Ranger: real 8.075s user 0.000s sys 0.009s
Sep 25, 2026 10:06:39 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 25, 2026 10:06:39 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Sep 25, 2026 10:06:39 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 25, 2026 10:06:39 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 25, 2026 10:06:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 25, 2026 10:06:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 25, 2026 10:06:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 25, 2026 10:06:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 25, 2026 10:06:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 25, 2026 10:06:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 25, 2026 10:06:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 25, 2026 10:06:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 25, 2026 10:06:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 25, 2026 10:06:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 25, 2026 10:06:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 25, 2026 10:06:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 25, 2026 10:06:39 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 25, 2026 10:06:39 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260925 22:06:40.506713 5121 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.5.0.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.5.0.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.5.0.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/rangerkms_127.5.0.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.5.0.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/rangerkms_127.5.0.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.0.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.5.0.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.0.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/HTTP_127.5.0.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.0.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/HTTP_127.5.0.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 25 22:06:40 dist-test-slave-7mqr krb5kdc[15869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374000, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260925 22:06:40.588272 5121 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260925 22:06:40.597525 5121 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260925 22:06:41.485862 5121 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260925 22:06:41.485946 5121 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ranger-kms
I20260925 22:06:41.485968 5121 mini_ranger_kms.cc:227] Using postgres at 127.5.0.84:45883
2026-09-25 22:06:41,661 [I] DB FLAVOR :POSTGRES
2026-09-25 22:06:41,661 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 22:06:41,662 [I] Checking connection
2026-09-25 22:06:41,662 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:45883/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-25 22:06:41,901 [I] connection success
2026-09-25 22:06:41,901 [I] --------- Verifying Ranger DB tables ---------
2026-09-25 22:06:41,901 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-25 22:06:41,902 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:45883/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-25 22:06:42,161 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-25 22:06:42,161 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 22:06:42,162 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-25 22:06:42,162 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:45883/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-25 22:06:42,417 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260925 22:06:42.836755 5121 mini_ranger_kms.cc:310] Created kms service
I20260925 22:06:43.000895 5121 mini_ranger_kms.cc:326] Created kudu user
I20260925 22:06:43.043592 5121 mini_ranger_kms.cc:343] Created rangerkms user
Sep 25 22:06:43 dist-test-slave-7mqr krb5kdc[15869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374000, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.5.0.84@KRBTEST.COM
I20260925 22:06:43.399012 5121 mini_ranger_kms.cc:384] Added ranger policy
I20260925 22:06:43.399132 5121 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ranger-kms:/tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260925 22:06:43.399156 5121 mini_ranger_kms.cc:240] Using host: 127.5.0.84
I20260925 22:06:43.402944 5121 mini_ranger_kms.cc:276] Ranger KMS PID: 16262
I20260925 22:06:43.403070 5121 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.5.0.84:59671
Sep 25, 2026 10:06:44 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 10:06:44 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 10:06:44 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 25, 2026 10:06:44 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 25, 2026 10:06:44 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ranger-kms/ews/webapp].
Sep 25, 2026 10:06:44 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260925 22:06:45.600112 5121 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.012s user 0.002s sys 0.156s
I20260925 22:06:45.600229 5121 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260925 22:06:45.600301 5121 mini_ranger_kms.cc:401] 127.5.0.84:59671/kms/v1/keys
Sep 25 22:06:45 dist-test-slave-7mqr krb5kdc[15869](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790374005, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.0.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 25 22:06:46 dist-test-slave-7mqr krb5kdc[15869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374005, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.0.84@KRBTEST.COM for HTTP/127.5.0.84@KRBTEST.COM
Sep 25 22:06:46 dist-test-slave-7mqr krb5kdc[15869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374005, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.0.84@KRBTEST.COM for HTTP/127.5.0.84@KRBTEST.COM
Sep 25 22:06:48 dist-test-slave-7mqr krb5kdc[15869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.5.0.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
I20260925 22:06:48.810220 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:34321
--webserver_interface=127.5.0.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:34249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:34321
--ranger_config_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.0.84:59671/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:06:48.892220 16313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:06:48.892412 16313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:06:48.892433 16313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:06:48.893584 16313 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:06:48.893626 16313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:06:48.893640 16313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:06:48.893652 16313 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:06:48.893700 16313 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:06:48.895205 16313 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:34249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/master-0/wal
--ranger_kms_url=127.5.0.84:59671/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:34321
--ranger_config_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:34321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:06:48.895447 16313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:06:48.895679 16313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:06:48.900642 16318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:06:48.900722 16313 server_base.cc:1034] running on GCE node
W20260925 22:06:48.900660 16319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:06:48.900652 16321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:06:48.901202 16313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:06:48.901495 16313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:06:48.902633 16313 hybrid_clock.cc:648] HybridClock initialized: now 1790374008902622 us; error 29 us; skew 500 ppm
Sep 25 22:06:48 dist-test-slave-7mqr krb5kdc[15869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374008, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:06:48.905053 16313 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:06:48.905494 16313 webserver.cc:460] Webserver started at http://127.5.0.126:41051/ using document root <none> and password file <none>
I20260925 22:06:48.905727 16313 fs_manager.cc:359] Metadata directory not provided
I20260925 22:06:48.905789 16313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:06:48.905944 16313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 22:06:48 dist-test-slave-7mqr krb5kdc[15869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374008, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for HTTP/127.5.0.84@KRBTEST.COM
I20260925 22:06:48.951854 16313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "1a17d35c45994df8822db588cf6dea29"
format_stamp: "Formatted at 2026-09-25 22:06:48 on dist-test-slave-7mqr"
server_key: "e514e142a9f93582b15cc38be7f93f5b"
server_key_iv: "503149c97163528fc88bff0a448ff897"
server_key_version: "kuduclusterkey@0"
I20260925 22:06:48.952422 16313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "1a17d35c45994df8822db588cf6dea29"
format_stamp: "Formatted at 2026-09-25 22:06:48 on dist-test-slave-7mqr"
server_key: "e514e142a9f93582b15cc38be7f93f5b"
server_key_iv: "503149c97163528fc88bff0a448ff897"
server_key_version: "kuduclusterkey@0"
I20260925 22:06:48.954027 16313 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:06:48.966445 16328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:48.966676 16313 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:06:48.966789 16313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "1a17d35c45994df8822db588cf6dea29"
format_stamp: "Formatted at 2026-09-25 22:06:48 on dist-test-slave-7mqr"
server_key: "e514e142a9f93582b15cc38be7f93f5b"
server_key_iv: "503149c97163528fc88bff0a448ff897"
server_key_version: "kuduclusterkey@0"
I20260925 22:06:48.966913 16313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:06:48.980527 16313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:06:48.982901 16313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:06:48.983086 16313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:06:48.988703 16313 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:34321
I20260925 22:06:48.988727 16380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:34321 every 8 connection(s)
I20260925 22:06:48.989131 16313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:06:48.989850 16381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:06:48.992347 16381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a17d35c45994df8822db588cf6dea29: Bootstrap starting.
I20260925 22:06:48.992998 16381 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a17d35c45994df8822db588cf6dea29: Neither blocks nor log segments found. Creating new log.
I20260925 22:06:48.993726 16381 log.cc:824] T 00000000000000000000000000000000 P 1a17d35c45994df8822db588cf6dea29: Log is configured to *not* fsync() on all Append() calls
I20260925 22:06:48.994488 16381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a17d35c45994df8822db588cf6dea29: No bootstrap required, opened a new log
I20260925 22:06:48.996191 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 16313
I20260925 22:06:48.996294 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:06:48.996483 5121 external_mini_cluster.cc:1428] Setting key cf3ecb6883d31fa89b76e9a1cdd31571
I20260925 22:06:48.997047 16381 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a17d35c45994df8822db588cf6dea29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a17d35c45994df8822db588cf6dea29" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34321 } }
I20260925 22:06:48.997298 16381 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a17d35c45994df8822db588cf6dea29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:06:48.997356 16381 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a17d35c45994df8822db588cf6dea29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a17d35c45994df8822db588cf6dea29, State: Initialized, Role: FOLLOWER
I20260925 22:06:48.997526 16381 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a17d35c45994df8822db588cf6dea29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a17d35c45994df8822db588cf6dea29" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34321 } }
I20260925 22:06:48.997726 16381 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a17d35c45994df8822db588cf6dea29 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:06:48.997787 16381 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a17d35c45994df8822db588cf6dea29 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:06:48.997843 16381 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a17d35c45994df8822db588cf6dea29 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:06:48.998687 16381 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a17d35c45994df8822db588cf6dea29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a17d35c45994df8822db588cf6dea29" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34321 } }
I20260925 22:06:48.998821 16381 leader_election.cc:302] T 00000000000000000000000000000000 P 1a17d35c45994df8822db588cf6dea29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a17d35c45994df8822db588cf6dea29; no voters:
I20260925 22:06:48.999025 16381 leader_election.cc:288] T 00000000000000000000000000000000 P 1a17d35c45994df8822db588cf6dea29 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:06:48.999054 16386 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a17d35c45994df8822db588cf6dea29 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:06:48.999277 16386 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a17d35c45994df8822db588cf6dea29 [term 1 LEADER]: Becoming Leader. State: Replica: 1a17d35c45994df8822db588cf6dea29, State: Running, Role: LEADER
I20260925 22:06:48.999384 16381 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a17d35c45994df8822db588cf6dea29 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:06:48.999408 16386 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a17d35c45994df8822db588cf6dea29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a17d35c45994df8822db588cf6dea29" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34321 } }
I20260925 22:06:48.999811 16386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a17d35c45994df8822db588cf6dea29 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a17d35c45994df8822db588cf6dea29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a17d35c45994df8822db588cf6dea29" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34321 } } }
I20260925 22:06:49.000032 16387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a17d35c45994df8822db588cf6dea29 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a17d35c45994df8822db588cf6dea29. Latest consensus state: current_term: 1 leader_uuid: "1a17d35c45994df8822db588cf6dea29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a17d35c45994df8822db588cf6dea29" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34321 } } }
I20260925 22:06:49.001152 16386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a17d35c45994df8822db588cf6dea29 [sys.catalog]: This master's current role is: LEADER
I20260925 22:06:49.001202 16387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a17d35c45994df8822db588cf6dea29 [sys.catalog]: This master's current role is: LEADER
I20260925 22:06:49.001343 16401 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:06:49.001801 16401 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:06:49.001052 16381 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Sep 25 22:06:49 dist-test-slave-7mqr krb5kdc[15869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:06:49.008891 16401 catalog_manager.cc:1347] Generated new cluster ID: 433f32611a3049beb055f418fb5ea008
I20260925 22:06:49.009003 16401 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:06:49.014119 16385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:48.997026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50230 (local address 127.5.0.126:34321)
0925 22:06:48.997216 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:48.997225 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:48.997296 (+ 71us) server_negotiation.cc:407] Connection header received
0925 22:06:48.997414 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:48.997421 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:48.997477 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:48.997608 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:06:48.998443 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:48.999861 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:49.001284 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:49.002057 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:49.002176 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:49.005328 (+ 3152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:06:49.005383 (+ 55us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:06:49.005384 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:06:49.005497 (+ 113us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:06:49.008803 (+ 3306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:06:49.012363 (+ 3560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:06:49.012367 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:06:49.012367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:06:49.012477 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:06:49.012666 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:06:49.012666 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:06:49.012667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:06:49.013018 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:06:49.013166 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0925 22:06:49.013691 (+ 525us) server_negotiation.cc:299] Negotiation successful
0925 22:06:49.013863 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":84,"threads_started":1}
I20260925 22:06:49.015930 16401 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:06:49.016762 16401 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:06:49.022538 16401 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a17d35c45994df8822db588cf6dea29: Generated new TSK 0
I20260925 22:06:49.022768 16401 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.0.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
I20260925 22:06:49.773007 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.0.84:59671/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.65:0
--local_ip_for_outbound_sockets=127.5.0.65
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:34321
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:34249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:06:49.879542 16426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:06:49.879791 16426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:06:49.879825 16426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:06:49.882277 16426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:06:49.882375 16426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:06:49.882431 16426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.65
I20260925 22:06:49.885377 16426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:34249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.5.0.84:59671/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:34321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.65
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:06:49.886011 16426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:06:49.886360 16426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:06:49.894230 16431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:06:49.894696 16434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:06:49.895205 16432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:06:49.895875 16426 server_base.cc:1034] running on GCE node
I20260925 22:06:49.896131 16426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:06:49.896412 16426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:06:49.899304 16426 hybrid_clock.cc:648] HybridClock initialized: now 1790374009899288 us; error 31 us; skew 500 ppm
Sep 25 22:06:49 dist-test-slave-7mqr krb5kdc[15869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374009, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:06:49.902949 16426 init.cc:377] Logged in from keytab as kudu/127.5.0.65@KRBTEST.COM (short username kudu)
I20260925 22:06:49.903592 16426 webserver.cc:460] Webserver started at http://127.5.0.65:45745/ using document root <none> and password file <none>
I20260925 22:06:49.903935 16426 fs_manager.cc:359] Metadata directory not provided
I20260925 22:06:49.904069 16426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:06:49.904278 16426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 22:06:49 dist-test-slave-7mqr krb5kdc[15869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374009, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for HTTP/127.5.0.84@KRBTEST.COM
I20260925 22:06:49.923463 16426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-0/data/instance:
uuid: "ba17b8e0019647c08bafd92f0d48b1cc"
format_stamp: "Formatted at 2026-09-25 22:06:49 on dist-test-slave-7mqr"
server_key: "24779f11406d97e7d10840dbbf6dd038"
server_key_iv: "80c318e1bb1b12e9bd0d93b841e83170"
server_key_version: "kuduclusterkey@0"
I20260925 22:06:49.924093 16426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance:
uuid: "ba17b8e0019647c08bafd92f0d48b1cc"
format_stamp: "Formatted at 2026-09-25 22:06:49 on dist-test-slave-7mqr"
server_key: "24779f11406d97e7d10840dbbf6dd038"
server_key_iv: "80c318e1bb1b12e9bd0d93b841e83170"
server_key_version: "kuduclusterkey@0"
I20260925 22:06:49.932364 16426 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.000s sys 0.002s
I20260925 22:06:49.953652 16443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:49.953927 16426 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:06:49.954030 16426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-0/wal
uuid: "ba17b8e0019647c08bafd92f0d48b1cc"
format_stamp: "Formatted at 2026-09-25 22:06:49 on dist-test-slave-7mqr"
server_key: "24779f11406d97e7d10840dbbf6dd038"
server_key_iv: "80c318e1bb1b12e9bd0d93b841e83170"
server_key_version: "kuduclusterkey@0"
I20260925 22:06:49.954128 16426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:06:49.975353 16426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:06:49.977941 16426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:06:49.978113 16426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:06:49.978406 16426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:06:49.978808 16426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:06:49.978859 16426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:49.978890 16426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:06:49.978948 16426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:49.986737 16426 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.65:45099
I20260925 22:06:49.986999 16556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.65:45099 every 8 connection(s)
I20260925 22:06:49.987186 16426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
I20260925 22:06:49.989333 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 16426
I20260925 22:06:49.989452 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance
I20260925 22:06:49.989668 5121 external_mini_cluster.cc:1428] Setting key 0e5db53b6a47bdcdfb226af19547fa12
Sep 25 22:06:49 dist-test-slave-7mqr krb5kdc[15869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374009, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:06:49.998013 16560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:49.988226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:33065 (local address 127.5.0.126:34321)
0925 22:06:49.988361 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:49.988363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:49.988581 (+ 218us) server_negotiation.cc:407] Connection header received
0925 22:06:49.989285 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:49.989286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:49.989297 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:49.989379 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:06:49.990360 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:49.991134 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:49.992207 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:49.992427 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:49.992501 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:49.995016 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:06:49.995017 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:06:49.995018 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:06:49.995026 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:06:49.996521 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:06:49.996947 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:06:49.996948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:06:49.996949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:06:49.997034 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:06:49.997321 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:06:49.997322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:06:49.997323 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:06:49.997452 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:06:49.997494 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0925 22:06:49.997874 (+ 380us) server_negotiation.cc:299] Negotiation successful
0925 22:06:49.997949 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":75,"threads_started":1}
I20260925 22:06:49.998463 16559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:49.988212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34321 (local address 127.5.0.65:33065)
0925 22:06:49.988431 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:49.988493 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0925 22:06:49.989160 (+ 667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:06:49.989538 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:06:49.989543 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:06:49.989792 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:06:49.990169 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:49.990176 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:49.991230 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:49.991231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:49.992113 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:49.992116 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:49.992519 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:49.992521 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:49.992636 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:49.993580 (+ 944us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:06:49.993588 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:06:49.994909 (+ 1321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:06:49.996623 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:06:49.996630 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:06:49.996632 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:06:49.996861 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:06:49.997103 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:06:49.997104 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:06:49.997104 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:06:49.997239 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:06:49.997542 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:06:49.997547 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:06:49.997717 (+ 170us) client_negotiation.cc:769] Sending connection context
0925 22:06:49.998126 (+ 409us) client_negotiation.cc:240] Negotiation successful
0925 22:06:49.998294 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":64,"threads_started":1}
I20260925 22:06:49.998984 16557 heartbeater.cc:347] Connected to a master server at 127.5.0.126:34321
I20260925 22:06:49.999089 16557 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.5.0.66@KRBTEST.COM; defaulting to no policy
I20260925 22:06:49.999292 16557 heartbeater.cc:511] Master 127.5.0.126:34321 requested a full tablet report, sending...
I20260925 22:06:49.999822 16345 ts_manager.cc:194] Registered new tserver with Master: ba17b8e0019647c08bafd92f0d48b1cc (127.5.0.65:45099)
I20260925 22:06:50.000651 16345 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.65@KRBTEST.COM'} at 127.5.0.65:33065
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
I20260925 22:06:50.019865 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.0.84:59671/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.66:0
--local_ip_for_outbound_sockets=127.5.0.66
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:34321
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:34249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 22:06:50 dist-test-slave-7mqr krb5kdc[15869](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790374010, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260925 22:06:50.146358 16563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:06:50.146607 16563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:06:50.146646 16563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:06:50.148239 16563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:06:50.148360 16563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:06:50.148444 16563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.66
I20260925 22:06:50.150300 16563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:34249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.5.0.84:59671/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:34321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.66
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:06:50.151345 16563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:06:50.151646 16563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:06:50.158201 16571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:06:50.158579 16570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:06:50.158799 16573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:06:50.159662 16563 server_base.cc:1034] running on GCE node
I20260925 22:06:50.159986 16563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:06:50.160254 16563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:06:50.161417 16563 hybrid_clock.cc:648] HybridClock initialized: now 1790374010161384 us; error 47 us; skew 500 ppm
Sep 25 22:06:50 dist-test-slave-7mqr krb5kdc[15869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374010, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:06:50.164664 16563 init.cc:377] Logged in from keytab as kudu/127.5.0.66@KRBTEST.COM (short username kudu)
I20260925 22:06:50.165220 16563 webserver.cc:460] Webserver started at http://127.5.0.66:34417/ using document root <none> and password file <none>
I20260925 22:06:50.165472 16563 fs_manager.cc:359] Metadata directory not provided
I20260925 22:06:50.165524 16563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:06:50.165663 16563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 22:06:50 dist-test-slave-7mqr krb5kdc[15869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374010, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for HTTP/127.5.0.84@KRBTEST.COM
I20260925 22:06:50.183722 16563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-1/data/instance:
uuid: "b3af946d5f6b4789905708120cb00598"
format_stamp: "Formatted at 2026-09-25 22:06:50 on dist-test-slave-7mqr"
server_key: "0ce1834b3f75e79ee5860ee6e87ba4ed"
server_key_iv: "0bded7facb3b80ff8c3117ba104693e3"
server_key_version: "kuduclusterkey@0"
I20260925 22:06:50.184206 16563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance:
uuid: "b3af946d5f6b4789905708120cb00598"
format_stamp: "Formatted at 2026-09-25 22:06:50 on dist-test-slave-7mqr"
server_key: "0ce1834b3f75e79ee5860ee6e87ba4ed"
server_key_iv: "0bded7facb3b80ff8c3117ba104693e3"
server_key_version: "kuduclusterkey@0"
I20260925 22:06:50.192874 16563 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.002s sys 0.000s
I20260925 22:06:50.206326 16580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:50.206607 16563 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 22:06:50.206710 16563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-1/wal
uuid: "b3af946d5f6b4789905708120cb00598"
format_stamp: "Formatted at 2026-09-25 22:06:50 on dist-test-slave-7mqr"
server_key: "0ce1834b3f75e79ee5860ee6e87ba4ed"
server_key_iv: "0bded7facb3b80ff8c3117ba104693e3"
server_key_version: "kuduclusterkey@0"
I20260925 22:06:50.206812 16563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:06:50.217646 16563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:06:50.220142 16563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:06:50.220357 16563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:06:50.220683 16563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:06:50.221086 16563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:06:50.221176 16563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:50.221259 16563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:06:50.221319 16563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:50.228803 16563 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.66:45881
I20260925 22:06:50.229318 16563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
I20260925 22:06:50.230199 16694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.66:45881 every 8 connection(s)
I20260925 22:06:50.236366 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 16563
I20260925 22:06:50.236496 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance
I20260925 22:06:50.236717 5121 external_mini_cluster.cc:1428] Setting key 26cba961155fcdb4cfac24ccc2518ec7
Sep 25 22:06:50 dist-test-slave-7mqr krb5kdc[15869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374010, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:06:50.248986 16560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:50.231173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:48593 (local address 127.5.0.126:34321)
0925 22:06:50.231237 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:50.231240 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:50.236186 (+ 4946us) server_negotiation.cc:407] Connection header received
0925 22:06:50.237053 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:50.237055 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:50.237071 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:50.237103 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:06:50.238122 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:50.238915 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:50.240450 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:50.240769 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:50.240850 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:50.245357 (+ 4507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:06:50.245359 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:06:50.245359 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:06:50.245369 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:06:50.247264 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:06:50.247781 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:06:50.247783 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:06:50.247783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:06:50.247844 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:06:50.248188 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:06:50.248189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:06:50.248189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:06:50.248334 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:06:50.248387 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 22:06:50.248782 (+ 395us) server_negotiation.cc:299] Negotiation successful
0925 22:06:50.248883 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 22:06:50.249780 16697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:50.230889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34321 (local address 127.5.0.66:48593)
0925 22:06:50.236000 (+ 5111us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:50.236065 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0925 22:06:50.236788 (+ 723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:06:50.237308 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:06:50.237313 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:06:50.237705 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:06:50.238004 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:50.238011 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:50.239043 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:50.239044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:50.240356 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:50.240358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:50.240923 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:50.240924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:50.241090 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:50.241753 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:06:50.241765 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:06:50.243570 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:06:50.247371 (+ 3801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:06:50.247379 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:06:50.247382 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:06:50.247688 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:06:50.247940 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:06:50.247941 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:06:50.247941 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:06:50.248104 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:06:50.248419 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:06:50.248425 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:06:50.248620 (+ 195us) client_negotiation.cc:769] Sending connection context
0925 22:06:50.249349 (+ 729us) client_negotiation.cc:240] Negotiation successful
0925 22:06:50.249526 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5043,"thread_start_us":81,"threads_started":1}
I20260925 22:06:50.252362 16695 heartbeater.cc:347] Connected to a master server at 127.5.0.126:34321
I20260925 22:06:50.252460 16695 heartbeater.cc:464] Registering TS with master...
I20260925 22:06:50.252647 16695 heartbeater.cc:511] Master 127.5.0.126:34321 requested a full tablet report, sending...
I20260925 22:06:50.253077 16345 ts_manager.cc:194] Registered new tserver with Master: b3af946d5f6b4789905708120cb00598 (127.5.0.66:45881)
I20260925 22:06:50.253582 16345 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.66@KRBTEST.COM'} at 127.5.0.66:48593
WARNING: no policy specified for kudu/127.5.0.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
I20260925 22:06:50.283047 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.0.84:59671/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.67:0
--local_ip_for_outbound_sockets=127.5.0.67
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:34321
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:34249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:06:50.400878 16700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:06:50.401117 16700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:06:50.401156 16700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:06:50.402552 16700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:06:50.402642 16700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:06:50.402696 16700 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.67
I20260925 22:06:50.404438 16700 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:34249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.5.0.84:59671/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:34321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.67
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:06:50.404713 16700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:06:50.404968 16700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:06:50.410270 16708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:06:50.410602 16706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:06:50.411607 16705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:06:50.411885 16700 server_base.cc:1034] running on GCE node
I20260925 22:06:50.412125 16700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:06:50.412385 16700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:06:50.413550 16700 hybrid_clock.cc:648] HybridClock initialized: now 1790374010413528 us; error 36 us; skew 500 ppm
Sep 25 22:06:50 dist-test-slave-7mqr krb5kdc[15869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374010, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:06:50.416827 16700 init.cc:377] Logged in from keytab as kudu/127.5.0.67@KRBTEST.COM (short username kudu)
I20260925 22:06:50.417359 16700 webserver.cc:460] Webserver started at http://127.5.0.67:39553/ using document root <none> and password file <none>
I20260925 22:06:50.417650 16700 fs_manager.cc:359] Metadata directory not provided
I20260925 22:06:50.417719 16700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:06:50.417852 16700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 22:06:50 dist-test-slave-7mqr krb5kdc[15869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374010, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for HTTP/127.5.0.84@KRBTEST.COM
I20260925 22:06:50.431669 16700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-2/data/instance:
uuid: "ff1c2600b4764e059f1c35044bbc2893"
format_stamp: "Formatted at 2026-09-25 22:06:50 on dist-test-slave-7mqr"
server_key: "83e62a2ba336f4f1c084c70ac1d04847"
server_key_iv: "8ff1582e7a0d476049d131f72117bcf6"
server_key_version: "kuduclusterkey@0"
I20260925 22:06:50.432175 16700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance:
uuid: "ff1c2600b4764e059f1c35044bbc2893"
format_stamp: "Formatted at 2026-09-25 22:06:50 on dist-test-slave-7mqr"
server_key: "83e62a2ba336f4f1c084c70ac1d04847"
server_key_iv: "8ff1582e7a0d476049d131f72117bcf6"
server_key_version: "kuduclusterkey@0"
I20260925 22:06:50.434372 16700 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260925 22:06:50.449599 16715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:50.449802 16700 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 22:06:50.449916 16700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-2/wal
uuid: "ff1c2600b4764e059f1c35044bbc2893"
format_stamp: "Formatted at 2026-09-25 22:06:50 on dist-test-slave-7mqr"
server_key: "83e62a2ba336f4f1c084c70ac1d04847"
server_key_iv: "8ff1582e7a0d476049d131f72117bcf6"
server_key_version: "kuduclusterkey@0"
I20260925 22:06:50.450037 16700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:06:50.462288 16700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:06:50.464570 16700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:06:50.464708 16700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:06:50.464968 16700 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:06:50.465312 16700 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:06:50.465345 16700 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:50.465366 16700 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:06:50.465411 16700 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:06:50.474059 16700 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.67:36563
I20260925 22:06:50.474469 16700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
I20260925 22:06:50.479053 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 16700
I20260925 22:06:50.479156 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance
I20260925 22:06:50.479355 5121 external_mini_cluster.cc:1428] Setting key a9cc0001891cdedbeaaeed20ebfa626d
I20260925 22:06:50.481015 16828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.67:36563 every 8 connection(s)
Sep 25 22:06:50 dist-test-slave-7mqr krb5kdc[15869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374010, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:06:50.491855 16560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:50.480645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:60513 (local address 127.5.0.126:34321)
0925 22:06:50.480695 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:50.480697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:50.481295 (+ 598us) server_negotiation.cc:407] Connection header received
0925 22:06:50.481882 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:50.481884 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:50.481896 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:50.481958 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:06:50.482702 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:50.483484 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:50.485015 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:50.485324 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:50.485365 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:50.487999 (+ 2634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:06:50.488001 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:06:50.488001 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:06:50.488009 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:06:50.489787 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:06:50.490287 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:06:50.490288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:06:50.490288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:06:50.490350 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:06:50.490975 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:06:50.490976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:06:50.490977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:06:50.491123 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:06:50.491168 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 22:06:50.491664 (+ 496us) server_negotiation.cc:299] Negotiation successful
0925 22:06:50.491754 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 22:06:50.492287 16831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:50.480739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34321 (local address 127.5.0.67:60513)
0925 22:06:50.481156 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:50.481195 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 22:06:50.481770 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:06:50.482061 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:06:50.482066 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:06:50.482312 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:06:50.482589 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:50.482595 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:50.483601 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:50.483602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:50.484899 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:50.484902 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:50.485452 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:50.485453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:50.485606 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:50.486239 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:06:50.486250 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:06:50.487855 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:06:50.489881 (+ 2026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:06:50.489887 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:06:50.489890 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:06:50.490200 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:06:50.490427 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:06:50.490428 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:06:50.490428 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:06:50.490764 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:06:50.491295 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:06:50.491302 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:06:50.491489 (+ 187us) client_negotiation.cc:769] Sending connection context
0925 22:06:50.491621 (+ 132us) client_negotiation.cc:240] Negotiation successful
0925 22:06:50.491779 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":185,"threads_started":1}
I20260925 22:06:50.493005 16829 heartbeater.cc:347] Connected to a master server at 127.5.0.126:34321
I20260925 22:06:50.493105 16829 heartbeater.cc:464] Registering TS with master...
I20260925 22:06:50.493291 16829 heartbeater.cc:511] Master 127.5.0.126:34321 requested a full tablet report, sending...
I20260925 22:06:50.493664 16345 ts_manager.cc:194] Registered new tserver with Master: ff1c2600b4764e059f1c35044bbc2893 (127.5.0.67:36563)
I20260925 22:06:50.494143 16345 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.67@KRBTEST.COM'} at 127.5.0.67:60513
I20260925 22:06:50.502121 5121 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:06:50.512233 16560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:50.503911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50234 (local address 127.5.0.126:34321)
0925 22:06:50.503964 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:50.503966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:50.503994 (+ 28us) server_negotiation.cc:407] Connection header received
0925 22:06:50.504068 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:50.504069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:50.504080 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:50.504152 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:06:50.504903 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:50.505669 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:50.506808 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:50.507112 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:50.507190 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:50.508823 (+ 1633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:06:50.508825 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:06:50.508825 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:06:50.508833 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:06:50.510743 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:06:50.511264 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:06:50.511265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:06:50.511266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:06:50.511332 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:06:50.511575 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:06:50.511576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:06:50.511576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:06:50.511717 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:06:50.511819 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 22:06:50.512055 (+ 236us) server_negotiation.cc:299] Negotiation successful
0925 22:06:50.512138 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 22:06:50.515870 16345 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50234:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:06:50.518263 16345 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:06:50.527321 16841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:50.521556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:36563 (local address 127.0.0.1:41124)
0925 22:06:50.521718 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:50.521765 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 22:06:50.521837 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:06:50.522781 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:06:50.522784 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:06:50.522786 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:06:50.522866 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:50.522868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:50.524237 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:50.524238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:50.525559 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:50.525562 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:50.527014 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:50.527015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:50.527109 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:50.527126 (+ 17us) client_negotiation.cc:769] Sending connection context
0925 22:06:50.527181 (+ 55us) client_negotiation.cc:240] Negotiation successful
0925 22:06:50.527237 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":82,"threads_started":1}
I20260925 22:06:50.527729 16843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:50.521813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:45099 (local address 127.0.0.1:60612)
0925 22:06:50.522830 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:50.522839 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:06:50.522918 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:06:50.523162 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:06:50.523162 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:06:50.523164 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:06:50.523203 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:50.523205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:50.524776 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:50.524776 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:50.525912 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:50.525915 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:50.527499 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:50.527500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:50.527563 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:50.527566 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:06:50.527615 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 22:06:50.527654 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":998,"thread_start_us":80,"threads_started":1}
I20260925 22:06:50.528033 16845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:50.522230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60612 (local address 127.5.0.65:45099)
0925 22:06:50.522361 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:50.522366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:50.522890 (+ 524us) server_negotiation.cc:407] Connection header received
0925 22:06:50.523012 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:50.523014 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:50.523030 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:50.523081 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:06:50.523280 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:50.524405 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:50.526163 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:50.527248 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:50.527298 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:50.527358 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 22:06:50.527912 (+ 554us) server_negotiation.cc:299] Negotiation successful
0925 22:06:50.527963 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":65,"threads_started":1}
I20260925 22:06:50.528345 16846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:50.522407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41124 (local address 127.5.0.67:36563)
0925 22:06:50.522588 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:50.522592 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:50.522603 (+ 11us) server_negotiation.cc:407] Connection header received
0925 22:06:50.522642 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:50.522644 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:50.522661 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:50.522947 (+ 286us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:06:50.523014 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:50.524132 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:50.526114 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:50.526910 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:50.527702 (+ 792us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:50.528062 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0925 22:06:50.528217 (+ 155us) server_negotiation.cc:299] Negotiation successful
0925 22:06:50.528261 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":84,"threads_started":1}
I20260925 22:06:50.528702 16491 tablet_service.cc:1467] Processing CreateTablet for tablet 9e7840c2d9fb4385b681703f49481bd7 (DEFAULT_TABLE table=test-table [id=d67443bd0cbd42468d50e3c35f750438]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:06:50.529100 16491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e7840c2d9fb4385b681703f49481bd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:06:50.532598 16763 tablet_service.cc:1467] Processing CreateTablet for tablet 9e7840c2d9fb4385b681703f49481bd7 (DEFAULT_TABLE table=test-table [id=d67443bd0cbd42468d50e3c35f750438]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:06:50.532969 16763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e7840c2d9fb4385b681703f49481bd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:06:50.533354 16847 tablet_bootstrap.cc:492] T 9e7840c2d9fb4385b681703f49481bd7 P ba17b8e0019647c08bafd92f0d48b1cc: Bootstrap starting.
I20260925 22:06:50.534050 16847 tablet_bootstrap.cc:654] T 9e7840c2d9fb4385b681703f49481bd7 P ba17b8e0019647c08bafd92f0d48b1cc: Neither blocks nor log segments found. Creating new log.
I20260925 22:06:50.534346 16847 log.cc:824] T 9e7840c2d9fb4385b681703f49481bd7 P ba17b8e0019647c08bafd92f0d48b1cc: Log is configured to *not* fsync() on all Append() calls
I20260925 22:06:50.539391 16842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:50.522028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:45881 (local address 127.0.0.1:32810)
0925 22:06:50.526038 (+ 4010us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:50.526048 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:06:50.526087 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:06:50.534873 (+ 8786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:06:50.534875 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:06:50.534877 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:06:50.534933 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:50.534935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:50.536896 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:50.536896 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:50.538071 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:50.538074 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:50.539110 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:50.539111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:50.539183 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:50.539188 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:06:50.539259 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 22:06:50.539307 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3994,"thread_start_us":63,"threads_started":1}
I20260925 22:06:50.539868 16844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:50.522135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32810 (local address 127.5.0.66:45881)
0925 22:06:50.534684 (+ 12549us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:50.534689 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:50.534703 (+ 14us) server_negotiation.cc:407] Connection header received
0925 22:06:50.534744 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:50.534746 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:50.534763 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:50.535019 (+ 256us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:06:50.535097 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:50.536801 (+ 1704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:50.538207 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:50.539003 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:50.539083 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:50.539163 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 22:06:50.539737 (+ 574us) server_negotiation.cc:299] Negotiation successful
0925 22:06:50.539782 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12502,"thread_start_us":94,"threads_started":1}
I20260925 22:06:50.540308 16847 tablet_bootstrap.cc:492] T 9e7840c2d9fb4385b681703f49481bd7 P ba17b8e0019647c08bafd92f0d48b1cc: No bootstrap required, opened a new log
I20260925 22:06:50.540391 16847 ts_tablet_manager.cc:1397] T 9e7840c2d9fb4385b681703f49481bd7 P ba17b8e0019647c08bafd92f0d48b1cc: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260925 22:06:50.540681 16626 tablet_service.cc:1467] Processing CreateTablet for tablet 9e7840c2d9fb4385b681703f49481bd7 (DEFAULT_TABLE table=test-table [id=d67443bd0cbd42468d50e3c35f750438]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:06:50.541020 16626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e7840c2d9fb4385b681703f49481bd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:06:50.542907 16847 raft_consensus.cc:348] T 9e7840c2d9fb4385b681703f49481bd7 P ba17b8e0019647c08bafd92f0d48b1cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1c2600b4764e059f1c35044bbc2893" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 36563 } } peers { permanent_uuid: "ba17b8e0019647c08bafd92f0d48b1cc" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45099 } } peers { permanent_uuid: "b3af946d5f6b4789905708120cb00598" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 45881 } }
I20260925 22:06:50.543105 16847 raft_consensus.cc:374] T 9e7840c2d9fb4385b681703f49481bd7 P ba17b8e0019647c08bafd92f0d48b1cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:06:50.543138 16847 raft_consensus.cc:729] T 9e7840c2d9fb4385b681703f49481bd7 P ba17b8e0019647c08bafd92f0d48b1cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba17b8e0019647c08bafd92f0d48b1cc, State: Initialized, Role: FOLLOWER
I20260925 22:06:50.543286 16847 consensus_queue.cc:260] T 9e7840c2d9fb4385b681703f49481bd7 P ba17b8e0019647c08bafd92f0d48b1cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1c2600b4764e059f1c35044bbc2893" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 36563 } } peers { permanent_uuid: "ba17b8e0019647c08bafd92f0d48b1cc" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45099 } } peers { permanent_uuid: "b3af946d5f6b4789905708120cb00598" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 45881 } }
I20260925 22:06:50.543556 16847 ts_tablet_manager.cc:1428] T 9e7840c2d9fb4385b681703f49481bd7 P ba17b8e0019647c08bafd92f0d48b1cc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 22:06:50.543679 16557 heartbeater.cc:503] Master 127.5.0.126:34321 was elected leader, sending a full tablet report...
I20260925 22:06:50.543845 16849 tablet_bootstrap.cc:492] T 9e7840c2d9fb4385b681703f49481bd7 P ff1c2600b4764e059f1c35044bbc2893: Bootstrap starting.
I20260925 22:06:50.544310 16851 tablet_bootstrap.cc:492] T 9e7840c2d9fb4385b681703f49481bd7 P b3af946d5f6b4789905708120cb00598: Bootstrap starting.
I20260925 22:06:50.545434 16851 tablet_bootstrap.cc:654] T 9e7840c2d9fb4385b681703f49481bd7 P b3af946d5f6b4789905708120cb00598: Neither blocks nor log segments found. Creating new log.
I20260925 22:06:50.545770 16849 tablet_bootstrap.cc:654] T 9e7840c2d9fb4385b681703f49481bd7 P ff1c2600b4764e059f1c35044bbc2893: Neither blocks nor log segments found. Creating new log.
I20260925 22:06:50.545897 16851 log.cc:824] T 9e7840c2d9fb4385b681703f49481bd7 P b3af946d5f6b4789905708120cb00598: Log is configured to *not* fsync() on all Append() calls
I20260925 22:06:50.546072 16849 log.cc:824] T 9e7840c2d9fb4385b681703f49481bd7 P ff1c2600b4764e059f1c35044bbc2893: Log is configured to *not* fsync() on all Append() calls
I20260925 22:06:50.547043 16849 tablet_bootstrap.cc:492] T 9e7840c2d9fb4385b681703f49481bd7 P ff1c2600b4764e059f1c35044bbc2893: No bootstrap required, opened a new log
I20260925 22:06:50.547113 16849 ts_tablet_manager.cc:1397] T 9e7840c2d9fb4385b681703f49481bd7 P ff1c2600b4764e059f1c35044bbc2893: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 22:06:50.549036 16849 raft_consensus.cc:348] T 9e7840c2d9fb4385b681703f49481bd7 P ff1c2600b4764e059f1c35044bbc2893 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1c2600b4764e059f1c35044bbc2893" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 36563 } } peers { permanent_uuid: "ba17b8e0019647c08bafd92f0d48b1cc" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45099 } } peers { permanent_uuid: "b3af946d5f6b4789905708120cb00598" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 45881 } }
I20260925 22:06:50.549201 16849 raft_consensus.cc:374] T 9e7840c2d9fb4385b681703f49481bd7 P ff1c2600b4764e059f1c35044bbc2893 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:06:50.549229 16849 raft_consensus.cc:729] T 9e7840c2d9fb4385b681703f49481bd7 P ff1c2600b4764e059f1c35044bbc2893 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff1c2600b4764e059f1c35044bbc2893, State: Initialized, Role: FOLLOWER
I20260925 22:06:50.549384 16849 consensus_queue.cc:260] T 9e7840c2d9fb4385b681703f49481bd7 P ff1c2600b4764e059f1c35044bbc2893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1c2600b4764e059f1c35044bbc2893" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 36563 } } peers { permanent_uuid: "ba17b8e0019647c08bafd92f0d48b1cc" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45099 } } peers { permanent_uuid: "b3af946d5f6b4789905708120cb00598" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 45881 } }
I20260925 22:06:50.549621 16849 ts_tablet_manager.cc:1428] T 9e7840c2d9fb4385b681703f49481bd7 P ff1c2600b4764e059f1c35044bbc2893: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 22:06:50.549722 16829 heartbeater.cc:503] Master 127.5.0.126:34321 was elected leader, sending a full tablet report...
I20260925 22:06:50.550885 16851 tablet_bootstrap.cc:492] T 9e7840c2d9fb4385b681703f49481bd7 P b3af946d5f6b4789905708120cb00598: No bootstrap required, opened a new log
I20260925 22:06:50.550962 16851 ts_tablet_manager.cc:1397] T 9e7840c2d9fb4385b681703f49481bd7 P b3af946d5f6b4789905708120cb00598: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260925 22:06:50.553385 16851 raft_consensus.cc:348] T 9e7840c2d9fb4385b681703f49481bd7 P b3af946d5f6b4789905708120cb00598 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1c2600b4764e059f1c35044bbc2893" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 36563 } } peers { permanent_uuid: "ba17b8e0019647c08bafd92f0d48b1cc" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45099 } } peers { permanent_uuid: "b3af946d5f6b4789905708120cb00598" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 45881 } }
I20260925 22:06:50.553578 16851 raft_consensus.cc:374] T 9e7840c2d9fb4385b681703f49481bd7 P b3af946d5f6b4789905708120cb00598 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:06:50.553627 16851 raft_consensus.cc:729] T 9e7840c2d9fb4385b681703f49481bd7 P b3af946d5f6b4789905708120cb00598 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3af946d5f6b4789905708120cb00598, State: Initialized, Role: FOLLOWER
I20260925 22:06:50.553736 16851 consensus_queue.cc:260] T 9e7840c2d9fb4385b681703f49481bd7 P b3af946d5f6b4789905708120cb00598 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1c2600b4764e059f1c35044bbc2893" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 36563 } } peers { permanent_uuid: "ba17b8e0019647c08bafd92f0d48b1cc" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45099 } } peers { permanent_uuid: "b3af946d5f6b4789905708120cb00598" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 45881 } }
I20260925 22:06:50.554014 16851 ts_tablet_manager.cc:1428] T 9e7840c2d9fb4385b681703f49481bd7 P b3af946d5f6b4789905708120cb00598: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 22:06:50.554284 16695 heartbeater.cc:503] Master 127.5.0.126:34321 was elected leader, sending a full tablet report...
Sep 25 22:06:50 dist-test-slave-7mqr krb5kdc[15869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374010, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for HTTP/127.5.0.84@KRBTEST.COM
I20260925 22:06:50.657220 16855 raft_consensus.cc:482] T 9e7840c2d9fb4385b681703f49481bd7 P b3af946d5f6b4789905708120cb00598 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:06:50.657364 16855 raft_consensus.cc:504] T 9e7840c2d9fb4385b681703f49481bd7 P b3af946d5f6b4789905708120cb00598 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1c2600b4764e059f1c35044bbc2893" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 36563 } } peers { permanent_uuid: "ba17b8e0019647c08bafd92f0d48b1cc" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45099 } } peers { permanent_uuid: "b3af946d5f6b4789905708120cb00598" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 45881 } }
I20260925 22:06:50.657763 16855 leader_election.cc:288] T 9e7840c2d9fb4385b681703f49481bd7 P b3af946d5f6b4789905708120cb00598 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff1c2600b4764e059f1c35044bbc2893 (127.5.0.67:36563), ba17b8e0019647c08bafd92f0d48b1cc (127.5.0.65:45099)
I20260925 22:06:50.661827 16697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:50.657945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:36563 (local address 127.5.0.66:37945)
0925 22:06:50.658010 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:50.658022 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:06:50.658063 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:06:50.658487 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:06:50.658488 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:06:50.658490 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:06:50.658569 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:50.658571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:50.659529 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:50.659531 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:50.660776 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:50.660779 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:50.661575 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:50.661576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:50.661656 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:50.661659 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:06:50.661706 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 22:06:50.661746 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260925 22:06:50.662307 16846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:50.658249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:37945 (local address 127.5.0.67:36563)
0925 22:06:50.658339 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:50.658341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:50.658350 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:06:50.658378 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:50.658379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:50.658389 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:50.658449 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:06:50.658665 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:50.659392 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:50.660890 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:50.661458 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:50.661530 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:50.661543 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0925 22:06:50.662194 (+ 651us) server_negotiation.cc:299] Negotiation successful
0925 22:06:50.662238 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 22:06:50.662608 16783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e7840c2d9fb4385b681703f49481bd7" candidate_uuid: "b3af946d5f6b4789905708120cb00598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff1c2600b4764e059f1c35044bbc2893" is_pre_election: true
I20260925 22:06:50.662761 16783 raft_consensus.cc:2454] T 9e7840c2d9fb4385b681703f49481bd7 P ff1c2600b4764e059f1c35044bbc2893 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3af946d5f6b4789905708120cb00598 in term 0.
I20260925 22:06:50.663328 16582 leader_election.cc:302] T 9e7840c2d9fb4385b681703f49481bd7 P b3af946d5f6b4789905708120cb00598 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3af946d5f6b4789905708120cb00598, ff1c2600b4764e059f1c35044bbc2893; no voters:
I20260925 22:06:50.663513 16855 raft_consensus.cc:2781] T 9e7840c2d9fb4385b681703f49481bd7 P b3af946d5f6b4789905708120cb00598 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:06:50.663594 16855 raft_consensus.cc:482] T 9e7840c2d9fb4385b681703f49481bd7 P b3af946d5f6b4789905708120cb00598 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:06:50.663620 16855 raft_consensus.cc:3037] T 9e7840c2d9fb4385b681703f49481bd7 P b3af946d5f6b4789905708120cb00598 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:06:50.663846 16845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:50.658296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:56913 (local address 127.5.0.65:45099)
0925 22:06:50.658509 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:50.658511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:50.658518 (+ 7us) server_negotiation.cc:407] Connection header received
0925 22:06:50.658541 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:50.658542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:50.658553 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:50.658599 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:06:50.659541 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:50.660297 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:50.662325 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:50.662916 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:50.662968 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:50.662977 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 22:06:50.663728 (+ 751us) server_negotiation.cc:299] Negotiation successful
0925 22:06:50.663776 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190}
I20260925 22:06:50.664114 16856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:50.657988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:45099 (local address 127.5.0.66:56913)
0925 22:06:50.658229 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:50.658238 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:06:50.658278 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:06:50.659336 (+ 1058us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:06:50.659337 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:06:50.659340 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:06:50.659453 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:50.659455 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:50.660391 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:50.660392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:50.661529 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:06:50.661531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:06:50.663051 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:06:50.663052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:06:50.663121 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:50.663124 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:06:50.663975 (+ 851us) client_negotiation.cc:240] Negotiation successful
0925 22:06:50.664019 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":107,"threads_started":1}
I20260925 22:06:50.664383 16511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e7840c2d9fb4385b681703f49481bd7" candidate_uuid: "b3af946d5f6b4789905708120cb00598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba17b8e0019647c08bafd92f0d48b1cc" is_pre_election: true
I20260925 22:06:50.664522 16511 raft_consensus.cc:2454] T 9e7840c2d9fb4385b681703f49481bd7 P ba17b8e0019647c08bafd92f0d48b1cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3af946d5f6b4789905708120cb00598 in term 0.
I20260925 22:06:50.665477 16855 raft_consensus.cc:504] T 9e7840c2d9fb4385b681703f49481bd7 P b3af946d5f6b4789905708120cb00598 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1c2600b4764e059f1c35044bbc2893" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 36563 } } peers { permanent_uuid: "ba17b8e0019647c08bafd92f0d48b1cc" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45099 } } peers { permanent_uuid: "b3af946d5f6b4789905708120cb00598" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 45881 } }
I20260925 22:06:50.665628 16855 leader_election.cc:288] T 9e7840c2d9fb4385b681703f49481bd7 P b3af946d5f6b4789905708120cb00598 [CANDIDATE]: Term 1 election: Requested vote from peers ff1c2600b4764e059f1c35044bbc2893 (127.5.0.67:36563), ba17b8e0019647c08bafd92f0d48b1cc (127.5.0.65:45099)
I20260925 22:06:50.665772 16511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e7840c2d9fb4385b681703f49481bd7" candidate_uuid: "b3af946d5f6b4789905708120cb00598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba17b8e0019647c08bafd92f0d48b1cc"
I20260925 22:06:50.665850 16511 raft_consensus.cc:3037] T 9e7840c2d9fb4385b681703f49481bd7 P ba17b8e0019647c08bafd92f0d48b1cc [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:06:50.665855 16783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e7840c2d9fb4385b681703f49481bd7" candidate_uuid: "b3af946d5f6b4789905708120cb00598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff1c2600b4764e059f1c35044bbc2893"
I20260925 22:06:50.665933 16783 raft_consensus.cc:3037] T 9e7840c2d9fb4385b681703f49481bd7 P ff1c2600b4764e059f1c35044bbc2893 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:06:50.666705 16511 raft_consensus.cc:2454] T 9e7840c2d9fb4385b681703f49481bd7 P ba17b8e0019647c08bafd92f0d48b1cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3af946d5f6b4789905708120cb00598 in term 1.
I20260925 22:06:50.666801 16783 raft_consensus.cc:2454] T 9e7840c2d9fb4385b681703f49481bd7 P ff1c2600b4764e059f1c35044bbc2893 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3af946d5f6b4789905708120cb00598 in term 1.
I20260925 22:06:50.666873 16584 leader_election.cc:302] T 9e7840c2d9fb4385b681703f49481bd7 P b3af946d5f6b4789905708120cb00598 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3af946d5f6b4789905708120cb00598, ba17b8e0019647c08bafd92f0d48b1cc; no voters:
I20260925 22:06:50.667223 16855 raft_consensus.cc:2781] T 9e7840c2d9fb4385b681703f49481bd7 P b3af946d5f6b4789905708120cb00598 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:06:50.667388 16855 raft_consensus.cc:686] T 9e7840c2d9fb4385b681703f49481bd7 P b3af946d5f6b4789905708120cb00598 [term 1 LEADER]: Becoming Leader. State: Replica: b3af946d5f6b4789905708120cb00598, State: Running, Role: LEADER
I20260925 22:06:50.667491 16855 consensus_queue.cc:237] T 9e7840c2d9fb4385b681703f49481bd7 P b3af946d5f6b4789905708120cb00598 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1c2600b4764e059f1c35044bbc2893" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 36563 } } peers { permanent_uuid: "ba17b8e0019647c08bafd92f0d48b1cc" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45099 } } peers { permanent_uuid: "b3af946d5f6b4789905708120cb00598" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 45881 } }
I20260925 22:06:50.668466 16345 catalog_manager.cc:5495] T 9e7840c2d9fb4385b681703f49481bd7 P b3af946d5f6b4789905708120cb00598 reported cstate change: term changed from 0 to 1, leader changed from <none> to b3af946d5f6b4789905708120cb00598 (127.5.0.66). New cstate: current_term: 1 leader_uuid: "b3af946d5f6b4789905708120cb00598" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1c2600b4764e059f1c35044bbc2893" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 36563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba17b8e0019647c08bafd92f0d48b1cc" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3af946d5f6b4789905708120cb00598" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 45881 } health_report { overall_health: HEALTHY } } }
I20260925 22:06:50.709652 16844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:50.705788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32814 (local address 127.5.0.66:45881)
0925 22:06:50.705828 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 22:06:50.705830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:06:50.705838 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:06:50.705877 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:06:50.705877 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:06:50.705900 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:06:50.705963 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:06:50.706153 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:50.706918 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:50.708109 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:06:50.708417 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:06:50.708487 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:06:50.708684 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:06:50.708814 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:06:50.708885 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 22:06:50.709543 (+ 658us) server_negotiation.cc:299] Negotiation successful
0925 22:06:50.709565 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 22:06:50.724527 16511 raft_consensus.cc:1261] T 9e7840c2d9fb4385b681703f49481bd7 P ba17b8e0019647c08bafd92f0d48b1cc [term 1 FOLLOWER]: Refusing update from remote peer b3af946d5f6b4789905708120cb00598: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:06:50.724910 16855 consensus_queue.cc:1035] T 9e7840c2d9fb4385b681703f49481bd7 P b3af946d5f6b4789905708120cb00598 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba17b8e0019647c08bafd92f0d48b1cc" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:06:50.728271 16783 raft_consensus.cc:1261] T 9e7840c2d9fb4385b681703f49481bd7 P ff1c2600b4764e059f1c35044bbc2893 [term 1 FOLLOWER]: Refusing update from remote peer b3af946d5f6b4789905708120cb00598: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:06:50.728576 16857 consensus_queue.cc:1035] T 9e7840c2d9fb4385b681703f49481bd7 P b3af946d5f6b4789905708120cb00598 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff1c2600b4764e059f1c35044bbc2893" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 36563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:06:50.728991 16860 mvcc.cc:205] Tried to move back new op lower bound from 7333371947910426624 to 7333371947694501888. Current Snapshot: MvccSnapshot[applied={T|T < 7333371947910426624}]
I20260925 22:06:50.734360 16345 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:06:50.734463 16345 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:06:50.735194 16345 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1a17d35c45994df8822db588cf6dea29: Sending DeleteTablet for 3 replicas of tablet 9e7840c2d9fb4385b681703f49481bd7
I20260925 22:06:50.735567 16491 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e7840c2d9fb4385b681703f49481bd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:06:50 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:60612
I20260925 22:06:50.735615 16763 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e7840c2d9fb4385b681703f49481bd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:06:50 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:41124
I20260925 22:06:50.736217 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 16426
I20260925 22:06:50.740115 16626 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e7840c2d9fb4385b681703f49481bd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:06:50 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:32810
I20260925 22:06:50.742815 16869 tablet_replica.cc:330] T 9e7840c2d9fb4385b681703f49481bd7 P ff1c2600b4764e059f1c35044bbc2893: stopping tablet replica
I20260925 22:06:50.743007 16869 raft_consensus.cc:2229] T 9e7840c2d9fb4385b681703f49481bd7 P ff1c2600b4764e059f1c35044bbc2893 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:06:50.743211 16869 raft_consensus.cc:2258] T 9e7840c2d9fb4385b681703f49481bd7 P ff1c2600b4764e059f1c35044bbc2893 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:06:50.743865 16869 ts_tablet_manager.cc:1905] T 9e7840c2d9fb4385b681703f49481bd7 P ff1c2600b4764e059f1c35044bbc2893: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:06:50.746371 16870 tablet_replica.cc:330] T 9e7840c2d9fb4385b681703f49481bd7 P b3af946d5f6b4789905708120cb00598: stopping tablet replica
I20260925 22:06:50.746589 16870 raft_consensus.cc:2229] T 9e7840c2d9fb4385b681703f49481bd7 P b3af946d5f6b4789905708120cb00598 [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:06:50.746812 16870 raft_consensus.cc:2258] T 9e7840c2d9fb4385b681703f49481bd7 P b3af946d5f6b4789905708120cb00598 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:06:50.747373 16870 ts_tablet_manager.cc:1905] T 9e7840c2d9fb4385b681703f49481bd7 P b3af946d5f6b4789905708120cb00598: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 22:06:50.747988 16332 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.0.65:45099 (error 108)
I20260925 22:06:50.748647 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 16563
I20260925 22:06:50.749311 16842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:50.748835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:45099 (local address 127.0.0.1:60628)
0925 22:06:50.749226 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:50.749266 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.65:45099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366}
I20260925 22:06:50.749392 16869 ts_tablet_manager.cc:1918] T 9e7840c2d9fb4385b681703f49481bd7 P ff1c2600b4764e059f1c35044bbc2893: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:06:50.749468 16869 log.cc:1196] T 9e7840c2d9fb4385b681703f49481bd7 P ff1c2600b4764e059f1c35044bbc2893: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373908987421-5121-0/minicluster-data/ts-2/wal/wals/9e7840c2d9fb4385b681703f49481bd7
I20260925 22:06:50.749755 16869 ts_tablet_manager.cc:1939] T 9e7840c2d9fb4385b681703f49481bd7 P ff1c2600b4764e059f1c35044bbc2893: Deleting consensus metadata
I20260925 22:06:50.750233 16330 catalog_manager.cc:4841] TS ff1c2600b4764e059f1c35044bbc2893 (127.5.0.67:36563): tablet 9e7840c2d9fb4385b681703f49481bd7 (table test-table [id=d67443bd0cbd42468d50e3c35f750438]) successfully deleted
W20260925 22:06:50.750293 16332 catalog_manager.cc:4584] TS ba17b8e0019647c08bafd92f0d48b1cc (127.5.0.65:45099): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9e7840c2d9fb4385b681703f49481bd7: Network error: Client connection negotiation failed: client connection to 127.5.0.65:45099: connect: Connection refused (error 111)
I20260925 22:06:50.754551 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 16700
I20260925 22:06:50.756124 16842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:06:50.754685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:45881 (local address 127.0.0.1:32830)
0925 22:06:50.756056 (+ 1371us) negotiation.cc:107] Waiting for socket to connect
0925 22:06:50.756089 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.66:45881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1348}
I20260925 22:06:50.759245 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 16313
I20260925 22:06:50.766049 5121 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260925 22:06:51.110478 5121 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260925 22:06:51.110579 5121 mini_ranger.cc:66] Stopping Ranger...
I20260925 22:06:51.455688 5121 mini_ranger.cc:68] Stopped Ranger
2026-09-25 22:06:51.455 UTC [15899] LOG: received smart shutdown request
2026-09-25 22:06:51.459 UTC [15899] LOG: background worker "logical replication launcher" (PID 15908) exited with exit code 1
2026-09-25 22:06:51.459 UTC [15903] LOG: shutting down
2026-09-25 22:06:51.635 UTC [15899] LOG: database system is shut down
2026-09-25T22:06:51Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (45080 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:06:51 dist-test-slave-7mqr krb5kdc[16880](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:06:51 dist-test-slave-7mqr krb5kdc[16880](info): set up 2 sockets
Sep 25 22:06:51 dist-test-slave-7mqr krb5kdc[16880](info): commencing operation
krb5kdc: starting...
W20260925 22:06:52.792171 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:06:52 dist-test-slave-7mqr krb5kdc[16880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:06:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:06:52Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.5.0.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.5.0.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.5.0.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/rangeradmin_127.5.0.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.5.0.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/rangeradmin_127.5.0.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.5.0.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.5.0.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.5.0.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/rangerlookup_127.5.0.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.5.0.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/rangerlookup_127.5.0.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.0.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.5.0.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.0.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/HTTP_127.5.0.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.0.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/HTTP_127.5.0.84@KRBTEST.COM.keytab.
I20260925 22:06:52.955236 5121 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskMOkht1/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/postgres -l logfile start
2026-09-25 22:06:55.456 UTC [16910] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-25 22:06:55.456 UTC [16910] LOG: listening on IPv4 address "127.5.0.84", port 51991
2026-09-25 22:06:55.460 UTC [16910] LOG: listening on Unix socket "/tmp/.s.PGSQL.51991"
2026-09-25 22:06:55.481 UTC [16913] LOG: database system was shut down at 2026-09-25 22:06:54 UTC
2026-09-25 22:06:55.484 UTC [16910] LOG: database system is ready to accept connections
I20260925 22:06:56.443743 5121 mini_postgres.cc:96] Postgres bound to 51991
2026-09-25 22:06:56.450 UTC [16921] FATAL: database "slave" does not exist
127.5.0.84:51991 - accepting connections
I20260925 22:06:56.450536 5121 mini_ranger.cc:161] Starting Ranger...
I20260925 22:06:56.464906 5121 mini_ranger.cc:84] Created miniranger Postgres user
I20260925 22:06:56.951596 5121 mini_ranger.cc:87] Created ranger Postgres database
I20260925 22:06:56.951690 5121 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ranger-admin
2026-09-25 22:06:57,220 [I] DB FLAVOR :POSTGRES
2026-09-25 22:06:57,220 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 22:06:57,220 [I] Checking connection..
2026-09-25 22:06:57,221 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:51991/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-25 22:06:57,481 [I] Checking connection passed.
2026-09-25 22:06:57,481 [I] --------- Verifying version history table ---------
2026-09-25 22:06:57,482 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:51991/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 22:06:57,711 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 22:06:57,711 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:51991/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 22:06:57,954 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 22:06:57,954 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-25 22:06:57,954 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:51991/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-25 22:06:58.196 UTC [16985] WARNING: there is no transaction in progress
2026-09-25 22:06:58,202 [I] create_dbversion_catalog.sql file imported successfully
2026-09-25 22:06:58,202 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:51991/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 22:06:58,445 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-25 22:06:58,445 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 22:06:58,446 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:51991/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-25 22:06:58,701 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:51991/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-25 22:06:58,927 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:51991/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-7mqr.c.gcp-upstream.internal','N') ;"
2026-09-25 22:06:59,179 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:51991/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 22:06:59,429 [I] Table x_portal_user does not exist in database ranger
2026-09-25 22:06:59,429 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:51991/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 22:06:59,692 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-25 22:06:59,692 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:51991/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 22:06:59,926 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:51991/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 22:07:00,186 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-25 22:07:00,186 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:51991/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-25 22:07:01.152 UTC [17093] WARNING: there is no transaction in progress
2026-09-25 22:07:01.168 UTC [17093] WARNING: there is no transaction in progress
2026-09-25 22:07:01.185 UTC [17093] WARNING: there is no transaction in progress
2026-09-25 22:07:01.201 UTC [17093] WARNING: there is no transaction in progress
2026-09-25 22:07:01.217 UTC [17093] WARNING: there is no transaction in progress
2026-09-25 22:07:01.236 UTC [17093] WARNING: there is no transaction in progress
2026-09-25 22:07:01.293 UTC [17093] WARNING: there is no transaction in progress
2026-09-25 22:07:01.300 UTC [17093] WARNING: there is no transaction in progress
2026-09-25 22:07:01.315 UTC [17093] WARNING: there is no transaction in progress
2026-09-25 22:07:01.327 UTC [17093] WARNING: there is no transaction in progress
2026-09-25 22:07:01.339 UTC [17093] WARNING: there is no transaction in progress
2026-09-25 22:07:01.352 UTC [17093] WARNING: there is no transaction in progress
2026-09-25 22:07:01.363 UTC [17093] WARNING: there is no transaction in progress
2026-09-25 22:07:01.370 UTC [17093] WARNING: there is no transaction in progress
2026-09-25 22:07:01.377 UTC [17093] WARNING: there is no transaction in progress
2026-09-25 22:07:01.384 UTC [17093] WARNING: there is no transaction in progress
2026-09-25 22:07:02,061 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-25 22:07:02,061 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:51991/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-25 22:07:02,318 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-25 22:07:02,318 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:51991/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 22:07:02,581 [I] Table x_portal_user already exists in database 'ranger'
2026-09-25 22:07:02,581 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:51991/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 22:07:02,832 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-25 22:07:02,833 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:51991/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 22:07:03,094 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:51991/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 22:07:03,322 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:51991/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-7mqr.c.gcp-upstream.internal';"
2026-09-25 22:07:03,548 [I] CORE_DB_SCHEMA import status has been updated
2026-09-25 22:07:03,548 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:51991/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-25 22:07:03,796 [I] DB_PATCHES have already been applied
I20260925 22:07:03.802273 5121 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ranger-admin:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskMOkht1/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260925 22:07:03.802379 5121 mini_ranger.cc:193] Using host: 127.5.0.84
I20260925 22:07:03.805359 5121 mini_ranger.cc:227] Ranger admin URL: http://127.5.0.84:53287
Sep 25, 2026 10:07:04 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 10:07:04 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 10:07:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ranger-admin/ews/webapp
Sep 25, 2026 10:07:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 25, 2026 10:07:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 25, 2026 10:07:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ranger-admin/ews/webapp].
Sep 25, 2026 10:07:04 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260925 22:07:05.002261 5121 mini_ranger.cc:160] Time spent starting Ranger: real 8.552s user 0.000s sys 0.009s
Sep 25, 2026 10:07:23 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 25, 2026 10:07:23 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 25, 2026 10:07:23 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 25, 2026 10:07:24 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 25, 2026 10:07:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 25, 2026 10:07:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 25, 2026 10:07:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 25, 2026 10:07:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 25, 2026 10:07:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 25, 2026 10:07:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 25, 2026 10:07:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 25, 2026 10:07:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 25, 2026 10:07:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 25, 2026 10:07:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 25, 2026 10:07:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 25, 2026 10:07:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 25, 2026 10:07:24 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 25, 2026 10:07:24 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260925 22:07:25.374915 5121 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.5.0.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.5.0.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.5.0.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/rangerkms_127.5.0.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.5.0.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/rangerkms_127.5.0.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.0.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.5.0.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.0.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/HTTP_127.5.0.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.0.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/HTTP_127.5.0.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 25 22:07:25 dist-test-slave-7mqr krb5kdc[16880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374045, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260925 22:07:25.453187 5121 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260925 22:07:25.462217 5121 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260925 22:07:26.309248 5121 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260925 22:07:26.309336 5121 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ranger-kms
I20260925 22:07:26.309355 5121 mini_ranger_kms.cc:227] Using postgres at 127.5.0.84:51991
2026-09-25 22:07:26,473 [I] DB FLAVOR :POSTGRES
2026-09-25 22:07:26,473 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 22:07:26,473 [I] Checking connection
2026-09-25 22:07:26,473 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:51991/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-25 22:07:26,731 [I] connection success
2026-09-25 22:07:26,731 [I] --------- Verifying Ranger DB tables ---------
2026-09-25 22:07:26,731 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-25 22:07:26,731 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:51991/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-25 22:07:26,966 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-25 22:07:26,967 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 22:07:26,967 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-25 22:07:26,967 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.0.84:51991/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-25 22:07:27,240 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260925 22:07:27.713608 5121 mini_ranger_kms.cc:310] Created kms service
I20260925 22:07:27.913712 5121 mini_ranger_kms.cc:326] Created kudu user
I20260925 22:07:27.955605 5121 mini_ranger_kms.cc:343] Created rangerkms user
Sep 25 22:07:27 dist-test-slave-7mqr krb5kdc[16880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374045, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.5.0.84@KRBTEST.COM
I20260925 22:07:28.327373 5121 mini_ranger_kms.cc:384] Added ranger policy
I20260925 22:07:28.327492 5121 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ranger-kms:/tmp/dist-test-taskMOkht1/build/release/bin/postgresql.jar:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskMOkht1/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260925 22:07:28.327512 5121 mini_ranger_kms.cc:240] Using host: 127.5.0.84
I20260925 22:07:28.330878 5121 mini_ranger_kms.cc:276] Ranger KMS PID: 17273
I20260925 22:07:28.330964 5121 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.5.0.84:46575
Sep 25, 2026 10:07:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 10:07:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 10:07:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 25, 2026 10:07:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 25, 2026 10:07:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ranger-kms/ews/webapp].
Sep 25, 2026 10:07:29 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260925 22:07:30.516261 5121 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.063s user 0.000s sys 0.146s
I20260925 22:07:30.516369 5121 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260925 22:07:30.516415 5121 mini_ranger_kms.cc:401] 127.5.0.84:46575/kms/v1/keys
Sep 25 22:07:30 dist-test-slave-7mqr krb5kdc[16880](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790374050, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.0.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 25 22:07:31 dist-test-slave-7mqr krb5kdc[16880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374050, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.0.84@KRBTEST.COM for HTTP/127.5.0.84@KRBTEST.COM
Sep 25 22:07:31 dist-test-slave-7mqr krb5kdc[16880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374050, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.0.84@KRBTEST.COM for HTTP/127.5.0.84@KRBTEST.COM
Sep 25 22:07:33 dist-test-slave-7mqr krb5kdc[16880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.5.0.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
I20260925 22:07:33.691583 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:36985
--webserver_interface=127.5.0.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:44899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:36985
--ranger_config_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.0.84:46575/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:07:33.808560 17324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:07:33.808835 17324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:07:33.808869 17324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:07:33.810479 17324 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:07:33.810534 17324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:07:33.810554 17324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:07:33.810571 17324 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:07:33.810588 17324 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:07:33.812870 17324 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:44899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/master-0/wal
--ranger_kms_url=127.5.0.84:46575/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:36985
--ranger_config_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:36985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:07:33.813133 17324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:07:33.813375 17324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:07:33.818786 17332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:33.818799 17330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:33.818804 17329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:07:33.819157 17324 server_base.cc:1034] running on GCE node
I20260925 22:07:33.819480 17324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:07:33.819804 17324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:07:33.820967 17324 hybrid_clock.cc:648] HybridClock initialized: now 1790374053820946 us; error 40 us; skew 500 ppm
Sep 25 22:07:33 dist-test-slave-7mqr krb5kdc[16880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374053, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:07:33.823596 17324 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:07:33.824112 17324 webserver.cc:460] Webserver started at http://127.5.0.126:43015/ using document root <none> and password file <none>
I20260925 22:07:33.824366 17324 fs_manager.cc:359] Metadata directory not provided
I20260925 22:07:33.824417 17324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:07:33.824537 17324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 22:07:33 dist-test-slave-7mqr krb5kdc[16880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374053, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for HTTP/127.5.0.84@KRBTEST.COM
I20260925 22:07:33.882906 17324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "b5c53f46f65344a487f29d36b547e6cd"
format_stamp: "Formatted at 2026-09-25 22:07:33 on dist-test-slave-7mqr"
server_key: "2945548ff0d6ccea03cfe34ad9007616"
server_key_iv: "76a12aaf4ac52360eeaa37f7863a018f"
server_key_version: "kuduclusterkey@0"
I20260925 22:07:33.883411 17324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "b5c53f46f65344a487f29d36b547e6cd"
format_stamp: "Formatted at 2026-09-25 22:07:33 on dist-test-slave-7mqr"
server_key: "2945548ff0d6ccea03cfe34ad9007616"
server_key_iv: "76a12aaf4ac52360eeaa37f7863a018f"
server_key_version: "kuduclusterkey@0"
I20260925 22:07:33.885020 17324 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:07:33.900126 17339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:33.900379 17324 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 22:07:33.900518 17324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "b5c53f46f65344a487f29d36b547e6cd"
format_stamp: "Formatted at 2026-09-25 22:07:33 on dist-test-slave-7mqr"
server_key: "2945548ff0d6ccea03cfe34ad9007616"
server_key_iv: "76a12aaf4ac52360eeaa37f7863a018f"
server_key_version: "kuduclusterkey@0"
I20260925 22:07:33.900638 17324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:07:33.910112 17324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:07:33.912305 17324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:07:33.912458 17324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:07:33.917577 17324 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:36985
I20260925 22:07:33.917604 17391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:36985 every 8 connection(s)
I20260925 22:07:33.918033 17324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:07:33.918583 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 17324
I20260925 22:07:33.918695 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:07:33.918919 17392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:07:33.918934 5121 external_mini_cluster.cc:1428] Setting key 036f7ea5dafce6c029e5c960f32a5c3c
I20260925 22:07:33.922367 17392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd: Bootstrap starting.
I20260925 22:07:33.923009 17392 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd: Neither blocks nor log segments found. Creating new log.
I20260925 22:07:33.923359 17392 log.cc:824] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd: Log is configured to *not* fsync() on all Append() calls
I20260925 22:07:33.924260 17392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd: No bootstrap required, opened a new log
I20260925 22:07:33.926805 17392 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5c53f46f65344a487f29d36b547e6cd" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 36985 } }
I20260925 22:07:33.926962 17392 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:07:33.927024 17392 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5c53f46f65344a487f29d36b547e6cd, State: Initialized, Role: FOLLOWER
Sep 25 22:07:33 dist-test-slave-7mqr krb5kdc[16880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:07:33.927187 17392 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5c53f46f65344a487f29d36b547e6cd" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 36985 } }
I20260925 22:07:33.927299 17392 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:07:33.927354 17392 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:07:33.927409 17392 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:07:33.928112 17392 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5c53f46f65344a487f29d36b547e6cd" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 36985 } }
I20260925 22:07:33.928237 17392 leader_election.cc:302] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5c53f46f65344a487f29d36b547e6cd; no voters:
I20260925 22:07:33.928395 17392 leader_election.cc:288] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:07:33.928421 17397 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:07:33.928629 17397 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [term 1 LEADER]: Becoming Leader. State: Replica: b5c53f46f65344a487f29d36b547e6cd, State: Running, Role: LEADER
I20260925 22:07:33.928728 17392 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:07:33.928750 17397 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5c53f46f65344a487f29d36b547e6cd" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 36985 } }
I20260925 22:07:33.929108 17399 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5c53f46f65344a487f29d36b547e6cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5c53f46f65344a487f29d36b547e6cd" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 36985 } } }
I20260925 22:07:33.929244 17399 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [sys.catalog]: This master's current role is: LEADER
I20260925 22:07:33.929502 17401 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:07:33.930060 17401 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:07:33.930490 17392 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 22:07:33.931193 17398 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5c53f46f65344a487f29d36b547e6cd. Latest consensus state: current_term: 1 leader_uuid: "b5c53f46f65344a487f29d36b547e6cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5c53f46f65344a487f29d36b547e6cd" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 36985 } } }
I20260925 22:07:33.931342 17398 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [sys.catalog]: This master's current role is: LEADER
I20260925 22:07:33.931604 17395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:33.919605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44424 (local address 127.5.0.126:36985)
0925 22:07:33.920008 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:33.920018 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:33.920053 (+ 35us) server_negotiation.cc:407] Connection header received
0925 22:07:33.920649 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:33.920658 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:33.920723 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:33.920871 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:33.921701 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:33.923129 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:33.924090 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:33.924724 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:33.924838 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:33.927735 (+ 2897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:33.927739 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:33.927739 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:33.927751 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:33.929853 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:33.930277 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:33.930280 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:33.930281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:33.930374 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:33.930580 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:33.930581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:33.930582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:33.930868 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:33.931001 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:33.931277 (+ 276us) server_negotiation.cc:299] Negotiation successful
0925 22:07:33.931399 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":67,"threads_started":1}
I20260925 22:07:33.933295 17401 catalog_manager.cc:1347] Generated new cluster ID: f09e4fc7ce2543d2bd140c075ee1f10f
I20260925 22:07:33.933338 17401 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:07:33.935241 17413 mvcc.cc:205] Tried to move back new op lower bound from 7333372124902080512 to 7333372124892680192. Current Snapshot: MvccSnapshot[applied={T|T < 7333372124902080512 or (T in {7333372124902080512})}]
I20260925 22:07:33.955886 17401 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:07:33.956507 17401 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:07:33.961257 17401 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd: Generated new TSK 0
I20260925 22:07:33.961448 17401 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.0.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
I20260925 22:07:34.618979 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.0.84:46575/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.65:0
--local_ip_for_outbound_sockets=127.5.0.65
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:36985
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:44899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:07:34.737594 17437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:07:34.737808 17437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:07:34.737833 17437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:07:34.739029 17437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:07:34.739073 17437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:07:34.739099 17437 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.65
I20260925 22:07:34.740685 17437 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:44899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.5.0.84:46575/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:36985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.65
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:07:34.740932 17437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:07:34.741186 17437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:07:34.746200 17443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:34.746181 17442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:34.746200 17445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:07:34.746385 17437 server_base.cc:1034] running on GCE node
I20260925 22:07:34.746726 17437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:07:34.746984 17437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:07:34.748131 17437 hybrid_clock.cc:648] HybridClock initialized: now 1790374054748109 us; error 38 us; skew 500 ppm
Sep 25 22:07:34 dist-test-slave-7mqr krb5kdc[16880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374054, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:07:34.750744 17437 init.cc:377] Logged in from keytab as kudu/127.5.0.65@KRBTEST.COM (short username kudu)
I20260925 22:07:34.751214 17437 webserver.cc:460] Webserver started at http://127.5.0.65:42197/ using document root <none> and password file <none>
I20260925 22:07:34.751459 17437 fs_manager.cc:359] Metadata directory not provided
I20260925 22:07:34.751539 17437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:07:34.751695 17437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 22:07:34 dist-test-slave-7mqr krb5kdc[16880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374054, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for HTTP/127.5.0.84@KRBTEST.COM
I20260925 22:07:34.763723 17437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-0/data/instance:
uuid: "4761cd7b68cf4c83b96e897c386d4b5a"
format_stamp: "Formatted at 2026-09-25 22:07:34 on dist-test-slave-7mqr"
server_key: "93672c94e224a60176eb13426b22842a"
server_key_iv: "c8d38e699499f60983956d4e29d1d853"
server_key_version: "kuduclusterkey@0"
I20260925 22:07:34.764292 17437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance:
uuid: "4761cd7b68cf4c83b96e897c386d4b5a"
format_stamp: "Formatted at 2026-09-25 22:07:34 on dist-test-slave-7mqr"
server_key: "93672c94e224a60176eb13426b22842a"
server_key_iv: "c8d38e699499f60983956d4e29d1d853"
server_key_version: "kuduclusterkey@0"
I20260925 22:07:34.765992 17437 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:07:34.778510 17452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:34.778720 17437 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:07:34.778816 17437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-0/wal
uuid: "4761cd7b68cf4c83b96e897c386d4b5a"
format_stamp: "Formatted at 2026-09-25 22:07:34 on dist-test-slave-7mqr"
server_key: "93672c94e224a60176eb13426b22842a"
server_key_iv: "c8d38e699499f60983956d4e29d1d853"
server_key_version: "kuduclusterkey@0"
I20260925 22:07:34.778930 17437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:07:34.791842 17437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:07:34.794493 17437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:07:34.794729 17437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:07:34.795043 17437 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:07:34.795428 17437 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:07:34.795490 17437 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:34.795557 17437 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:07:34.795599 17437 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:34.802536 17437 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.65:41819
I20260925 22:07:34.803067 17437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
I20260925 22:07:34.804201 17567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.65:41819 every 8 connection(s)
I20260925 22:07:34.805358 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 17437
I20260925 22:07:34.805450 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance
I20260925 22:07:34.805678 5121 external_mini_cluster.cc:1428] Setting key b94d06bec80e8c2b5cc139684108ae00
Sep 25 22:07:34 dist-test-slave-7mqr krb5kdc[16880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374054, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:07:34.818367 17570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:34.805710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:36985 (local address 127.5.0.65:46655)
0925 22:07:34.806135 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:34.806181 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:34.808160 (+ 1979us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:34.808614 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:34.808619 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:34.808826 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:07:34.809020 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:34.809024 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:34.810039 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:34.810040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:34.811440 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:34.811443 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:34.812132 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:34.812133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:34.812292 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:34.812920 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:07:34.812933 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:07:34.814591 (+ 1658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:07:34.816724 (+ 2133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:34.816731 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:34.816733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:34.817018 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:34.817246 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:34.817247 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:34.817247 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:34.817372 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:34.817662 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:07:34.817667 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:07:34.817845 (+ 178us) client_negotiation.cc:769] Sending connection context
0925 22:07:34.817955 (+ 110us) client_negotiation.cc:240] Negotiation successful
0925 22:07:34.818122 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":88,"threads_started":1}
I20260925 22:07:34.818830 17571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:34.805989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:46655 (local address 127.5.0.126:36985)
0925 22:07:34.808356 (+ 2367us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:34.808358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:34.808366 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:07:34.808392 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:34.808393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:34.808405 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:34.808466 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:34.809118 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:34.809936 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:34.811547 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:34.811958 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:34.812027 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:34.814697 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:34.814699 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:34.814699 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:34.814707 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:34.816630 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:34.817115 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:34.817116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:34.817117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:34.817181 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:34.817440 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:34.817441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:34.817441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:34.817578 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:34.817636 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:34.818641 (+ 1005us) server_negotiation.cc:299] Negotiation successful
0925 22:07:34.818730 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2333,"thread_start_us":80,"threads_started":1}
I20260925 22:07:34.819365 17568 heartbeater.cc:347] Connected to a master server at 127.5.0.126:36985
I20260925 22:07:34.819459 17568 heartbeater.cc:464] Registering TS with master...
I20260925 22:07:34.819643 17568 heartbeater.cc:511] Master 127.5.0.126:36985 requested a full tablet report, sending...
I20260925 22:07:34.820161 17356 ts_manager.cc:194] Registered new tserver with Master: 4761cd7b68cf4c83b96e897c386d4b5a (127.5.0.65:41819)
I20260925 22:07:34.820780 17356 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.65@KRBTEST.COM'} at 127.5.0.65:46655
WARNING: no policy specified for kudu/127.5.0.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
I20260925 22:07:34.846763 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.0.84:46575/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.66:0
--local_ip_for_outbound_sockets=127.5.0.66
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:36985
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:44899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:07:34.966598 17574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:07:34.966876 17574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:07:34.966913 17574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:07:34.968618 17574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:07:34.968683 17574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:07:34.968724 17574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.66
I20260925 22:07:34.970901 17574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:44899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.5.0.84:46575/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:36985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.66
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:07:34.971148 17574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:07:34.971469 17574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:07:34.978132 17582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:07:34.978262 17574 server_base.cc:1034] running on GCE node
W20260925 22:07:34.978417 17581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:34.978538 17584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:07:34.978802 17574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:07:34.979094 17574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:07:34.980263 17574 hybrid_clock.cc:648] HybridClock initialized: now 1790374054980241 us; error 33 us; skew 500 ppm
Sep 25 22:07:34 dist-test-slave-7mqr krb5kdc[16880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374054, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:07:34.983397 17574 init.cc:377] Logged in from keytab as kudu/127.5.0.66@KRBTEST.COM (short username kudu)
I20260925 22:07:34.983945 17574 webserver.cc:460] Webserver started at http://127.5.0.66:42575/ using document root <none> and password file <none>
I20260925 22:07:34.984220 17574 fs_manager.cc:359] Metadata directory not provided
I20260925 22:07:34.984279 17574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:07:34.984448 17574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 22:07:34 dist-test-slave-7mqr krb5kdc[16880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374054, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for HTTP/127.5.0.84@KRBTEST.COM
Sep 25 22:07:34 dist-test-slave-7mqr krb5kdc[16880](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790374054, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:07:35.002894 17574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-1/data/instance:
uuid: "dff6386c354f4c4cb813d458441a2d1b"
format_stamp: "Formatted at 2026-09-25 22:07:34 on dist-test-slave-7mqr"
server_key: "006d04ccbefc1d40fdc7fae39b0f2850"
server_key_iv: "3d04d2db39b2c5f297987b4f276a4fbf"
server_key_version: "kuduclusterkey@0"
I20260925 22:07:35.003384 17574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance:
uuid: "dff6386c354f4c4cb813d458441a2d1b"
format_stamp: "Formatted at 2026-09-25 22:07:34 on dist-test-slave-7mqr"
server_key: "006d04ccbefc1d40fdc7fae39b0f2850"
server_key_iv: "3d04d2db39b2c5f297987b4f276a4fbf"
server_key_version: "kuduclusterkey@0"
I20260925 22:07:35.008299 17574 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260925 22:07:35.030882 17591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:35.032796 17574 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.001s sys 0.000s
I20260925 22:07:35.032920 17574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-1/wal
uuid: "dff6386c354f4c4cb813d458441a2d1b"
format_stamp: "Formatted at 2026-09-25 22:07:34 on dist-test-slave-7mqr"
server_key: "006d04ccbefc1d40fdc7fae39b0f2850"
server_key_iv: "3d04d2db39b2c5f297987b4f276a4fbf"
server_key_version: "kuduclusterkey@0"
I20260925 22:07:35.033028 17574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:07:35.048982 17574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:07:35.051636 17574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:07:35.051937 17574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:07:35.052203 17574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:07:35.052591 17574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:07:35.052644 17574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:35.052712 17574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:07:35.052742 17574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:35.059962 17574 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.66:33067
I20260925 22:07:35.060387 17574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
I20260925 22:07:35.061050 17704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.66:33067 every 8 connection(s)
I20260925 22:07:35.064620 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 17574
I20260925 22:07:35.064750 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance
I20260925 22:07:35.064965 5121 external_mini_cluster.cc:1428] Setting key 2a472ee694d6376ad7edd0c9b125027a
Sep 25 22:07:35 dist-test-slave-7mqr krb5kdc[16880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374054, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:07:35.070909 17571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:35.061450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:44571 (local address 127.5.0.126:36985)
0925 22:07:35.061493 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:35.061495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:35.061506 (+ 11us) server_negotiation.cc:407] Connection header received
0925 22:07:35.061968 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:35.061970 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:35.061982 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:35.062039 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:35.062584 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:35.063106 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:35.064214 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:35.064511 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:35.064565 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:35.067411 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:35.067413 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:35.067413 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:35.067422 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:35.069251 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:35.069761 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:35.069762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:35.069762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:35.069829 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:35.070120 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:35.070121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:35.070121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:35.070263 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:35.070307 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:35.070714 (+ 407us) server_negotiation.cc:299] Negotiation successful
0925 22:07:35.070811 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 22:07:35.071653 17707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:35.061126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:36985 (local address 127.5.0.66:44571)
0925 22:07:35.061268 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:35.061310 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:35.061847 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:35.062112 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:35.062116 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:35.062290 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:07:35.062486 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:35.062490 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:35.063204 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:35.063205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:35.064107 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:35.064110 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:35.064675 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:35.064676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:35.064956 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:35.065594 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:07:35.065604 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:07:35.067299 (+ 1695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:07:35.069350 (+ 2051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:35.069355 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:35.069358 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:35.069671 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:35.069903 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:35.069904 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:35.069904 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:35.070034 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:35.070356 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:07:35.070363 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:07:35.070547 (+ 184us) client_negotiation.cc:769] Sending connection context
0925 22:07:35.071246 (+ 699us) client_negotiation.cc:240] Negotiation successful
0925 22:07:35.071412 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":85,"threads_started":1}
I20260925 22:07:35.072359 17705 heartbeater.cc:347] Connected to a master server at 127.5.0.126:36985
I20260925 22:07:35.072512 17705 heartbeater.cc:464] Registering TS with master...
I20260925 22:07:35.072741 17705 heartbeater.cc:511] Master 127.5.0.126:36985 requested a full tablet report, sending...
I20260925 22:07:35.073145 17356 ts_manager.cc:194] Registered new tserver with Master: dff6386c354f4c4cb813d458441a2d1b (127.5.0.66:33067)
I20260925 22:07:35.073603 17356 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.66@KRBTEST.COM'} at 127.5.0.66:44571
WARNING: no policy specified for kudu/127.5.0.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
I20260925 22:07:35.103264 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.0.84:46575/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.67:0
--local_ip_for_outbound_sockets=127.5.0.67
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:36985
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:44899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:07:35.209355 17711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:07:35.209695 17711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:07:35.209784 17711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:07:35.211529 17711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:07:35.211622 17711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:07:35.211700 17711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.67
I20260925 22:07:35.213855 17711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:44899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.5.0.84:46575/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:36985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.67
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:07:35.214290 17711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:07:35.214576 17711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:07:35.224582 17717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:35.225078 17716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:35.225400 17719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:07:35.226518 17711 server_base.cc:1034] running on GCE node
I20260925 22:07:35.226732 17711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:07:35.227034 17711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:07:35.240005 17711 hybrid_clock.cc:648] HybridClock initialized: now 1790374055239974 us; error 51 us; skew 500 ppm
Sep 25 22:07:35 dist-test-slave-7mqr krb5kdc[16880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374055, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:07:35.247053 17711 init.cc:377] Logged in from keytab as kudu/127.5.0.67@KRBTEST.COM (short username kudu)
I20260925 22:07:35.247591 17711 webserver.cc:460] Webserver started at http://127.5.0.67:37917/ using document root <none> and password file <none>
I20260925 22:07:35.247830 17711 fs_manager.cc:359] Metadata directory not provided
I20260925 22:07:35.247875 17711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:07:35.248031 17711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 22:07:35 dist-test-slave-7mqr krb5kdc[16880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374055, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for HTTP/127.5.0.84@KRBTEST.COM
I20260925 22:07:35.263116 17711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-2/data/instance:
uuid: "c639c7d469cb4ab59c886ca28bdc1624"
format_stamp: "Formatted at 2026-09-25 22:07:35 on dist-test-slave-7mqr"
server_key: "8d8a0c5d14ed84b3db92d652787a9c48"
server_key_iv: "0567b98309ee61bfa4cb34c2934cb188"
server_key_version: "kuduclusterkey@0"
I20260925 22:07:35.263674 17711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance:
uuid: "c639c7d469cb4ab59c886ca28bdc1624"
format_stamp: "Formatted at 2026-09-25 22:07:35 on dist-test-slave-7mqr"
server_key: "8d8a0c5d14ed84b3db92d652787a9c48"
server_key_iv: "0567b98309ee61bfa4cb34c2934cb188"
server_key_version: "kuduclusterkey@0"
I20260925 22:07:35.266760 17711 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.001s
I20260925 22:07:35.283007 17726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:35.283221 17711 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:07:35.283306 17711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-2/wal
uuid: "c639c7d469cb4ab59c886ca28bdc1624"
format_stamp: "Formatted at 2026-09-25 22:07:35 on dist-test-slave-7mqr"
server_key: "8d8a0c5d14ed84b3db92d652787a9c48"
server_key_iv: "0567b98309ee61bfa4cb34c2934cb188"
server_key_version: "kuduclusterkey@0"
I20260925 22:07:35.283390 17711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:07:35.328118 17711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:07:35.331808 17711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:07:35.332079 17711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:07:35.332418 17711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:07:35.332852 17711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:07:35.332955 17711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:35.333029 17711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:07:35.333089 17711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:35.349568 17711 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.67:34539
I20260925 22:07:35.350136 17711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
I20260925 22:07:35.350445 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 17711
I20260925 22:07:35.350566 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance
I20260925 22:07:35.350766 5121 external_mini_cluster.cc:1428] Setting key a7a026773ec7ae99f1b8fc785250b662
I20260925 22:07:35.351109 17839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.67:34539 every 8 connection(s)
Sep 25 22:07:35 dist-test-slave-7mqr krb5kdc[16880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374055, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:07:35.363083 17571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:35.352105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:59437 (local address 127.5.0.126:36985)
0925 22:07:35.352152 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:35.352154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:35.352504 (+ 350us) server_negotiation.cc:407] Connection header received
0925 22:07:35.353214 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:35.353215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:35.353227 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:35.353258 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:35.354105 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:35.354883 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:35.356428 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:35.356730 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:35.356782 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:35.359531 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:35.359533 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:35.359533 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:35.359540 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:35.361435 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:35.361946 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:35.361948 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:35.361948 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:35.362014 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:35.362308 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:35.362308 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:35.362309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:35.362444 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:35.362490 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:35.362887 (+ 397us) server_negotiation.cc:299] Negotiation successful
0925 22:07:35.362984 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 22:07:35.363958 17842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:35.351797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:36985 (local address 127.5.0.67:59437)
0925 22:07:35.352374 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:35.352419 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:35.353086 (+ 667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:35.353445 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:35.353449 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:35.353699 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:07:35.353992 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:35.353999 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:35.355016 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:35.355017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:35.356278 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:35.356281 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:35.356889 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:35.356890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:35.357060 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:35.357654 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:07:35.357666 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:07:35.359425 (+ 1759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:07:35.361529 (+ 2104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:35.361535 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:35.361538 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:35.361832 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:35.362092 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:35.362093 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:35.362093 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:35.362228 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:35.362540 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:07:35.362547 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:07:35.362725 (+ 178us) client_negotiation.cc:769] Sending connection context
0925 22:07:35.363513 (+ 788us) client_negotiation.cc:240] Negotiation successful
0925 22:07:35.363680 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":117,"threads_started":1}
I20260925 22:07:35.364601 17840 heartbeater.cc:347] Connected to a master server at 127.5.0.126:36985
I20260925 22:07:35.364696 17840 heartbeater.cc:464] Registering TS with master...
I20260925 22:07:35.364871 17840 heartbeater.cc:511] Master 127.5.0.126:36985 requested a full tablet report, sending...
I20260925 22:07:35.365283 17356 ts_manager.cc:194] Registered new tserver with Master: c639c7d469cb4ab59c886ca28bdc1624 (127.5.0.67:34539)
I20260925 22:07:35.365765 17356 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.67@KRBTEST.COM'} at 127.5.0.67:59437
I20260925 22:07:35.373093 5121 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:07:35.373842 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 17437
I20260925 22:07:35.379745 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 17574
I20260925 22:07:35.386315 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 17711
I20260925 22:07:35.392831 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 17324
I20260925 22:07:35.399266 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:36985
--webserver_interface=127.5.0.126
--webserver_port=43015
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:44899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:36985
--ranger_config_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.0.84:46575/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.0.84:46575/kms with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:07:35.481199 17843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:07:35.481415 17843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:07:35.481437 17843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:07:35.482570 17843 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:07:35.482615 17843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:07:35.482630 17843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:07:35.482642 17843 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:07:35.482695 17843 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:07:35.484359 17843 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:44899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.0.84:46575/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:36985
--ranger_config_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:36985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=43015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:07:35.484591 17843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:07:35.484833 17843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 22:07:35.489632 17843 server_base.cc:1034] running on GCE node
W20260925 22:07:35.489657 17851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:35.489705 17849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:35.489707 17848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:07:35.490099 17843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:07:35.490340 17843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:07:35.491528 17843 hybrid_clock.cc:648] HybridClock initialized: now 1790374055491517 us; error 32 us; skew 500 ppm
Sep 25 22:07:35 dist-test-slave-7mqr krb5kdc[16880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374055, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:07:35.493974 17843 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:07:35.494381 17843 webserver.cc:460] Webserver started at http://127.5.0.126:43015/ using document root <none> and password file <none>
I20260925 22:07:35.494635 17843 fs_manager.cc:359] Metadata directory not provided
I20260925 22:07:35.494711 17843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 22:07:35 dist-test-slave-7mqr krb5kdc[16880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374055, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for HTTP/127.5.0.84@KRBTEST.COM
I20260925 22:07:35.556164 17843 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:07:35.557005 17859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:35.557204 17843 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:07:35.557315 17843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "b5c53f46f65344a487f29d36b547e6cd"
format_stamp: "Formatted at 2026-09-25 22:07:33 on dist-test-slave-7mqr"
server_key: "2945548ff0d6ccea03cfe34ad9007616"
server_key_iv: "76a12aaf4ac52360eeaa37f7863a018f"
server_key_version: "kuduclusterkey@0"
I20260925 22:07:35.557794 17843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:07:35.574055 17843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:07:35.575802 17843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:07:35.575989 17843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:07:35.580147 17843 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:36985
I20260925 22:07:35.580192 17911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:36985 every 8 connection(s)
I20260925 22:07:35.580554 17843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:07:35.581764 17912 sys_catalog.cc:263] Verifying existing consensus state
I20260925 22:07:35.582406 17912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd: Bootstrap starting.
I20260925 22:07:35.585291 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 17843
I20260925 22:07:35.585624 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.0.84:46575/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.65:41819
--local_ip_for_outbound_sockets=127.5.0.65
--tserver_master_addrs=127.5.0.126:36985
--webserver_port=42197
--webserver_interface=127.5.0.65
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:44899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.0.84:46575/kms with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:07:35.590874 17912 log.cc:824] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd: Log is configured to *not* fsync() on all Append() calls
I20260925 22:07:35.592581 17912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 22:07:35.592867 17912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd: Bootstrap complete.
I20260925 22:07:35.595206 17912 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5c53f46f65344a487f29d36b547e6cd" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 36985 } }
I20260925 22:07:35.595428 17912 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5c53f46f65344a487f29d36b547e6cd, State: Initialized, Role: FOLLOWER
I20260925 22:07:35.595606 17912 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5c53f46f65344a487f29d36b547e6cd" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 36985 } }
I20260925 22:07:35.595674 17912 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:07:35.595732 17912 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:07:35.595777 17912 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [term 1 FOLLOWER]: Advancing to term 2
I20260925 22:07:35.596786 17912 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5c53f46f65344a487f29d36b547e6cd" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 36985 } }
I20260925 22:07:35.596871 17912 leader_election.cc:302] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5c53f46f65344a487f29d36b547e6cd; no voters:
I20260925 22:07:35.597081 17912 leader_election.cc:288] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 22:07:35.597115 17916 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [term 2 FOLLOWER]: Leader election won for term 2
I20260925 22:07:35.597220 17916 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [term 2 LEADER]: Becoming Leader. State: Replica: b5c53f46f65344a487f29d36b547e6cd, State: Running, Role: LEADER
I20260925 22:07:35.597355 17912 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:07:35.597402 17916 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5c53f46f65344a487f29d36b547e6cd" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 36985 } }
I20260925 22:07:35.597744 17917 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b5c53f46f65344a487f29d36b547e6cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5c53f46f65344a487f29d36b547e6cd" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 36985 } } }
I20260925 22:07:35.597743 17916 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5c53f46f65344a487f29d36b547e6cd. Latest consensus state: current_term: 2 leader_uuid: "b5c53f46f65344a487f29d36b547e6cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5c53f46f65344a487f29d36b547e6cd" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 36985 } } }
I20260925 22:07:35.597839 17917 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [sys.catalog]: This master's current role is: LEADER
I20260925 22:07:35.597867 17916 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd [sys.catalog]: This master's current role is: LEADER
I20260925 22:07:35.598090 17922 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:07:35.598698 17922 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:07:35.598851 17922 catalog_manager.cc:1259] Loaded cluster ID: f09e4fc7ce2543d2bd140c075ee1f10f
I20260925 22:07:35.598912 17922 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:07:35.599109 17912 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 22:07:35.602442 17922 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:07:35.603027 17922 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd: Loaded TSK: 0
I20260925 22:07:35.603439 17922 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 22:07:35.697585 17914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:07:35.697880 17914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:07:35.697917 17914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:07:35.699766 17914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:07:35.699824 17914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:07:35.699864 17914 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.65
I20260925 22:07:35.702165 17914 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:44899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.0.84:46575/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.65:41819
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.0.65
--webserver_port=42197
--tserver_master_addrs=127.5.0.126:36985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.65
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:07:35.702426 17914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:07:35.702728 17914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:07:35.709406 17952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:35.709685 17950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:07:35.709910 17914 server_base.cc:1034] running on GCE node
W20260925 22:07:35.710489 17949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:07:35.710717 17914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:07:35.710999 17914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:07:35.712162 17914 hybrid_clock.cc:648] HybridClock initialized: now 1790374055712133 us; error 44 us; skew 500 ppm
Sep 25 22:07:35 dist-test-slave-7mqr krb5kdc[16880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374055, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:07:35.715440 17914 init.cc:377] Logged in from keytab as kudu/127.5.0.65@KRBTEST.COM (short username kudu)
I20260925 22:07:35.716077 17914 webserver.cc:460] Webserver started at http://127.5.0.65:42197/ using document root <none> and password file <none>
I20260925 22:07:35.716351 17914 fs_manager.cc:359] Metadata directory not provided
I20260925 22:07:35.716434 17914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 22:07:35 dist-test-slave-7mqr krb5kdc[16880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374055, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for HTTP/127.5.0.84@KRBTEST.COM
I20260925 22:07:35.748543 17914 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:07:35.749616 17960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:35.749794 17914 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:07:35.749873 17914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-0/wal
uuid: "4761cd7b68cf4c83b96e897c386d4b5a"
format_stamp: "Formatted at 2026-09-25 22:07:34 on dist-test-slave-7mqr"
server_key: "93672c94e224a60176eb13426b22842a"
server_key_iv: "c8d38e699499f60983956d4e29d1d853"
server_key_version: "kuduclusterkey@0"
I20260925 22:07:35.750363 17914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:07:35.764285 17914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:07:35.766819 17914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:07:35.767014 17914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:07:35.767339 17914 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:07:35.767741 17914 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:07:35.767798 17914 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:35.767831 17914 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:07:35.767938 17914 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:35.776427 17914 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.65:41819
I20260925 22:07:35.776640 18073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.65:41819 every 8 connection(s)
I20260925 22:07:35.776947 17914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
I20260925 22:07:35.781786 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 17914
I20260925 22:07:35.782109 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.0.84:46575/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.66:33067
--local_ip_for_outbound_sockets=127.5.0.66
--tserver_master_addrs=127.5.0.126:36985
--webserver_port=42575
--webserver_interface=127.5.0.66
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:44899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.0.84:46575/kms with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 22:07:35 dist-test-slave-7mqr krb5kdc[16880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374055, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:07:35.796639 18077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:35.777861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:33317 (local address 127.5.0.126:36985)
0925 22:07:35.778075 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:35.778083 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:35.778127 (+ 44us) server_negotiation.cc:407] Connection header received
0925 22:07:35.778741 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:35.778746 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:35.778791 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:35.778925 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:35.779475 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:35.780596 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:35.781606 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:35.782097 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:35.782222 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:35.784819 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:35.784823 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:35.784824 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:35.784835 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:35.786912 (+ 2077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:35.788001 (+ 1089us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:35.788004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:35.788005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:35.788118 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:35.793777 (+ 5659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:35.793778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:35.793778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:35.794130 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:35.794235 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:35.796248 (+ 2013us) server_negotiation.cc:299] Negotiation successful
0925 22:07:35.796401 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":88,"threads_started":1}
I20260925 22:07:35.798615 18076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:35.777758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:36985 (local address 127.5.0.65:33317)
0925 22:07:35.777941 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:35.777987 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:35.778499 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:35.778962 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:35.778966 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:35.779172 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:07:35.779358 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:35.779362 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:35.780706 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:35.780707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:35.781500 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:35.781501 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:35.782402 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:35.782403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:35.782504 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:35.783054 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:07:35.783062 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:07:35.784696 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:07:35.787008 (+ 2312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:35.787014 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:35.787017 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:35.787919 (+ 902us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:35.788202 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:35.788203 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:35.788203 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:35.793569 (+ 5366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:35.794223 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:07:35.794231 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:07:35.795736 (+ 1505us) client_negotiation.cc:769] Sending connection context
0925 22:07:35.798200 (+ 2464us) client_negotiation.cc:240] Negotiation successful
0925 22:07:35.798380 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":105,"threads_started":1}
I20260925 22:07:35.799368 18074 heartbeater.cc:347] Connected to a master server at 127.5.0.126:36985
I20260925 22:07:35.799470 18074 heartbeater.cc:464] Registering TS with master...
I20260925 22:07:35.799671 18074 heartbeater.cc:511] Master 127.5.0.126:36985 requested a full tablet report, sending...
W20260925 22:07:35.800619 18074 heartbeater.cc:644] Failed to heartbeat to 127.5.0.126:36985 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 22:07:35.800693 18074 heartbeater.cc:464] Registering TS with master...
I20260925 22:07:35.800773 18074 heartbeater.cc:511] Master 127.5.0.126:36985 requested a full tablet report, sending...
I20260925 22:07:35.801213 18074 heartbeater.cc:464] Registering TS with master...
I20260925 22:07:35.801287 18074 heartbeater.cc:511] Master 127.5.0.126:36985 requested a full tablet report, sending...
W20260925 22:07:35.801652 18074 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 22:07:35.889647 18078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:07:35.889926 18078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:07:35.889971 18078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:07:35.891681 18078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:07:35.891741 18078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:07:35.891779 18078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.66
I20260925 22:07:35.893757 18078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:44899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.0.84:46575/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.66:33067
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.0.66
--webserver_port=42575
--tserver_master_addrs=127.5.0.126:36985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.66
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:07:35.894063 18078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:07:35.894336 18078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:07:35.900667 18086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:35.900996 18084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:35.901196 18083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:07:35.901262 18078 server_base.cc:1034] running on GCE node
I20260925 22:07:35.901543 18078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:07:35.901830 18078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:07:35.903044 18078 hybrid_clock.cc:648] HybridClock initialized: now 1790374055903017 us; error 43 us; skew 500 ppm
Sep 25 22:07:35 dist-test-slave-7mqr krb5kdc[16880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374055, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:07:35.906253 18078 init.cc:377] Logged in from keytab as kudu/127.5.0.66@KRBTEST.COM (short username kudu)
I20260925 22:07:35.906694 18078 webserver.cc:460] Webserver started at http://127.5.0.66:42575/ using document root <none> and password file <none>
I20260925 22:07:35.906941 18078 fs_manager.cc:359] Metadata directory not provided
I20260925 22:07:35.907025 18078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 22:07:35 dist-test-slave-7mqr krb5kdc[16880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374055, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for HTTP/127.5.0.84@KRBTEST.COM
I20260925 22:07:35.927428 18078 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:07:35.928645 18094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:35.928869 18078 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:07:35.928967 18078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-1/wal
uuid: "dff6386c354f4c4cb813d458441a2d1b"
format_stamp: "Formatted at 2026-09-25 22:07:34 on dist-test-slave-7mqr"
server_key: "006d04ccbefc1d40fdc7fae39b0f2850"
server_key_iv: "3d04d2db39b2c5f297987b4f276a4fbf"
server_key_version: "kuduclusterkey@0"
I20260925 22:07:35.929411 18078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:07:35.949775 18078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:07:35.952029 18078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:07:35.952191 18078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:07:35.952456 18078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:07:35.952777 18078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:07:35.952833 18078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:35.952875 18078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:07:35.952903 18078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:35.960260 18078 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.66:33067
I20260925 22:07:35.960359 18207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.66:33067 every 8 connection(s)
I20260925 22:07:35.960693 18078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
I20260925 22:07:35.963800 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 18078
I20260925 22:07:35.964131 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.0.84:46575/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.67:34539
--local_ip_for_outbound_sockets=127.5.0.67
--tserver_master_addrs=127.5.0.126:36985
--webserver_port=37917
--webserver_interface=127.5.0.67
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:44899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.0.84:46575/kms with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 22:07:35 dist-test-slave-7mqr krb5kdc[16880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374055, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:07:35.972116 18077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:35.961776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:39957 (local address 127.5.0.126:36985)
0925 22:07:35.961825 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:35.961827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:35.961837 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:07:35.962296 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:35.962297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:35.962312 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:35.962358 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:35.963338 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:35.964056 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:35.965615 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:35.965938 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:35.965982 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:35.968434 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:35.968436 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:35.968447 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:35.968455 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:35.970300 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:35.970836 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:35.970837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:35.970838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:35.970899 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:35.971229 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:35.971230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:35.971231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:35.971382 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:35.971430 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:35.971845 (+ 415us) server_negotiation.cc:299] Negotiation successful
0925 22:07:35.971966 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 22:07:35.972967 18210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:35.961475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:36985 (local address 127.5.0.66:39957)
0925 22:07:35.961627 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:35.961669 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:35.962187 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:35.962730 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:35.962733 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:35.962977 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:07:35.963251 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:35.963258 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:35.964164 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:35.964165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:35.965503 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:35.965507 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:35.966098 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:35.966099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:35.966239 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:35.966830 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:07:35.966848 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:07:35.968330 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:07:35.970397 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:35.970405 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:35.970407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:35.970744 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:35.970998 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:35.970999 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:35.970999 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:35.971145 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:35.971497 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:07:35.971503 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:07:35.971677 (+ 174us) client_negotiation.cc:769] Sending connection context
0925 22:07:35.972499 (+ 822us) client_negotiation.cc:240] Negotiation successful
0925 22:07:35.972681 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":84,"threads_started":1}
I20260925 22:07:35.973678 18208 heartbeater.cc:347] Connected to a master server at 127.5.0.126:36985
I20260925 22:07:35.973774 18208 heartbeater.cc:464] Registering TS with master...
I20260925 22:07:35.973958 18208 heartbeater.cc:511] Master 127.5.0.126:36985 requested a full tablet report, sending...
W20260925 22:07:35.974634 18208 heartbeater.cc:644] Failed to heartbeat to 127.5.0.126:36985 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 22:07:35.974717 18208 heartbeater.cc:464] Registering TS with master...
I20260925 22:07:35.974782 18208 heartbeater.cc:511] Master 127.5.0.126:36985 requested a full tablet report, sending...
I20260925 22:07:35.975091 18208 heartbeater.cc:464] Registering TS with master...
I20260925 22:07:35.975173 18208 heartbeater.cc:511] Master 127.5.0.126:36985 requested a full tablet report, sending...
W20260925 22:07:35.975541 18208 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 22:07:36.068009 18211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:07:36.068270 18211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:07:36.068311 18211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:07:36.069978 18211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:07:36.070039 18211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:07:36.070083 18211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.67
I20260925 22:07:36.072269 18211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:44899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.0.84:46575/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.67:34539
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.0.67
--webserver_port=37917
--tserver_master_addrs=127.5.0.126:36985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.67
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:07:36.072538 18211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:07:36.072793 18211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:07:36.079381 18219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:36.079668 18217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:36.079700 18216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:07:36.080087 18211 server_base.cc:1034] running on GCE node
I20260925 22:07:36.080314 18211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:07:36.080607 18211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:07:36.081801 18211 hybrid_clock.cc:648] HybridClock initialized: now 1790374056081785 us; error 26 us; skew 500 ppm
Sep 25 22:07:36 dist-test-slave-7mqr krb5kdc[16880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374056, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:07:36.085703 18211 init.cc:377] Logged in from keytab as kudu/127.5.0.67@KRBTEST.COM (short username kudu)
I20260925 22:07:36.086321 18211 webserver.cc:460] Webserver started at http://127.5.0.67:37917/ using document root <none> and password file <none>
I20260925 22:07:36.086658 18211 fs_manager.cc:359] Metadata directory not provided
I20260925 22:07:36.086756 18211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 22:07:36 dist-test-slave-7mqr krb5kdc[16880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374056, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for HTTP/127.5.0.84@KRBTEST.COM
I20260925 22:07:36.104293 18211 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260925 22:07:36.108361 18228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:36.108589 18211 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.000s
I20260925 22:07:36.108675 18211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-2/wal
uuid: "c639c7d469cb4ab59c886ca28bdc1624"
format_stamp: "Formatted at 2026-09-25 22:07:35 on dist-test-slave-7mqr"
server_key: "8d8a0c5d14ed84b3db92d652787a9c48"
server_key_iv: "0567b98309ee61bfa4cb34c2934cb188"
server_key_version: "kuduclusterkey@0"
I20260925 22:07:36.109138 18211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:07:36.129673 18211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:07:36.132277 18211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:07:36.132476 18211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:07:36.132719 18211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:07:36.133204 18211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:07:36.133266 18211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:36.133308 18211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:07:36.133334 18211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:36.141865 18211 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.67:34539
I20260925 22:07:36.142369 18211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
I20260925 22:07:36.144171 18341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.67:34539 every 8 connection(s)
Sep 25 22:07:36 dist-test-slave-7mqr krb5kdc[16880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374056, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:07:36.152351 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 18211
I20260925 22:07:36.153532 18077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:36.143228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:42891 (local address 127.5.0.126:36985)
0925 22:07:36.143284 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:36.143287 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:36.143543 (+ 256us) server_negotiation.cc:407] Connection header received
0925 22:07:36.144265 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:36.144267 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:36.144284 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:36.144320 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:36.145158 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:36.145923 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:36.147369 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:36.147682 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:36.147772 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:36.150273 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:36.150275 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:36.150275 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:36.150283 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:36.151842 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:36.152368 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:36.152369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:36.152369 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:36.152435 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:36.152774 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:36.152775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:36.152776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:36.152899 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:36.153006 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:36.153357 (+ 351us) server_negotiation.cc:299] Negotiation successful
0925 22:07:36.153443 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 22:07:36.153728 18344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:36.143254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:36985 (local address 127.5.0.67:42891)
0925 22:07:36.143412 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:36.143459 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:36.144110 (+ 651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:36.144515 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:36.144519 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:36.144711 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:07:36.145039 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:36.145046 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:36.146039 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:36.146040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:36.147273 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:36.147276 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:36.147882 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:36.147883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:36.148071 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:36.148711 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:07:36.148719 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:07:36.150175 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:07:36.151966 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:36.151974 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:36.151981 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:36.152264 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:36.152519 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:36.152520 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:36.152520 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:36.152670 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:36.153010 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:07:36.153017 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:07:36.153205 (+ 188us) client_negotiation.cc:769] Sending connection context
0925 22:07:36.153335 (+ 130us) client_negotiation.cc:240] Negotiation successful
0925 22:07:36.153483 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
I20260925 22:07:36.154181 18342 heartbeater.cc:347] Connected to a master server at 127.5.0.126:36985
I20260925 22:07:36.154281 18342 heartbeater.cc:464] Registering TS with master...
I20260925 22:07:36.154467 18342 heartbeater.cc:511] Master 127.5.0.126:36985 requested a full tablet report, sending...
W20260925 22:07:36.155210 18342 heartbeater.cc:644] Failed to heartbeat to 127.5.0.126:36985 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 22:07:36.155287 18342 heartbeater.cc:464] Registering TS with master...
I20260925 22:07:36.155352 18342 heartbeater.cc:511] Master 127.5.0.126:36985 requested a full tablet report, sending...
I20260925 22:07:36.156456 18342 heartbeater.cc:464] Registering TS with master...
I20260925 22:07:36.156536 18342 heartbeater.cc:511] Master 127.5.0.126:36985 requested a full tablet report, sending...
W20260925 22:07:36.156858 18342 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 22:07:36.161300 18346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:36.152987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44434 (local address 127.5.0.126:36985)
0925 22:07:36.153093 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:36.153095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:36.153101 (+ 6us) server_negotiation.cc:407] Connection header received
0925 22:07:36.153119 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:36.153120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:36.153131 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:36.153196 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:36.153980 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:36.155560 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:36.156437 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:36.156740 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:36.156821 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:36.158039 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:36.158040 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:36.158041 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:36.158047 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:36.160152 (+ 2105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:36.160499 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:36.160500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:36.160501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:36.160569 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:36.160782 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:36.160783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:36.160783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:36.160889 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:36.161078 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:36.161116 (+ 38us) server_negotiation.cc:299] Negotiation successful
0925 22:07:36.161196 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":60,"threads_started":1}
Sep 25 22:07:36 dist-test-slave-7mqr krb5kdc[16880](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790374056, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:07:36.801844 18074 heartbeater.cc:464] Registering TS with master...
I20260925 22:07:36.802013 18074 heartbeater.cc:511] Master 127.5.0.126:36985 requested a full tablet report, sending...
I20260925 22:07:36.802362 17876 ts_manager.cc:194] Registered new tserver with Master: 4761cd7b68cf4c83b96e897c386d4b5a (127.5.0.65:41819)
I20260925 22:07:36.803470 17876 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.65@KRBTEST.COM'} at 127.5.0.65:33317
I20260925 22:07:36.975724 18208 heartbeater.cc:464] Registering TS with master...
I20260925 22:07:36.975880 18208 heartbeater.cc:511] Master 127.5.0.126:36985 requested a full tablet report, sending...
I20260925 22:07:36.976238 17876 ts_manager.cc:194] Registered new tserver with Master: dff6386c354f4c4cb813d458441a2d1b (127.5.0.66:33067)
I20260925 22:07:36.976778 17876 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.66@KRBTEST.COM'} at 127.5.0.66:39957
Sep 25 22:07:37 dist-test-slave-7mqr krb5kdc[16880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374056, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for HTTP/127.5.0.84@KRBTEST.COM
I20260925 22:07:37.157034 18342 heartbeater.cc:464] Registering TS with master...
I20260925 22:07:37.157202 18342 heartbeater.cc:511] Master 127.5.0.126:36985 requested a full tablet report, sending...
I20260925 22:07:37.157604 17876 ts_manager.cc:194] Registered new tserver with Master: c639c7d469cb4ab59c886ca28bdc1624 (127.5.0.67:34539)
I20260925 22:07:37.158035 17876 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.67@KRBTEST.COM'} at 127.5.0.67:42891
I20260925 22:07:37.160020 5121 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:07:37.167642 18364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:37.160989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44448 (local address 127.5.0.126:36985)
0925 22:07:37.161117 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:37.161119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:37.161173 (+ 54us) server_negotiation.cc:407] Connection header received
0925 22:07:37.161244 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:37.161245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:37.161256 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:37.161319 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:37.161909 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:37.162555 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:37.163401 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:37.163623 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:37.163698 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:37.164918 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:37.164920 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:37.164920 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:37.164927 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:37.166476 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:37.166844 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:37.166845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:37.166846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:37.166909 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:37.167120 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:37.167120 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:37.167121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:37.167230 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:37.167329 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:37.167444 (+ 115us) server_negotiation.cc:299] Negotiation successful
0925 22:07:37.167523 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":68,"threads_started":1}
I20260925 22:07:37.169564 17876 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44448:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:07:37.171352 17876 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:07:37.179504 18371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:37.174368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:34539 (local address 127.0.0.1:45588)
0925 22:07:37.174757 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:37.174801 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:37.174869 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:37.175618 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:37.175621 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:37.175624 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:07:37.175691 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:37.175693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:37.176933 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:37.176933 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:37.178157 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:37.178159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:37.179230 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:37.179231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:37.179311 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:37.179328 (+ 17us) client_negotiation.cc:769] Sending connection context
0925 22:07:37.179376 (+ 48us) client_negotiation.cc:240] Negotiation successful
0925 22:07:37.179425 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":94,"threads_started":1}
I20260925 22:07:37.180154 18373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:37.175065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45588 (local address 127.5.0.67:34539)
0925 22:07:37.175451 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:37.175455 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:37.175466 (+ 11us) server_negotiation.cc:407] Connection header received
0925 22:07:37.175503 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:37.175505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:37.175521 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:37.175600 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:07:37.175750 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:37.176847 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:37.178385 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:37.179133 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:37.179948 (+ 815us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:37.180011 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:37.180037 (+ 26us) server_negotiation.cc:299] Negotiation successful
0925 22:07:37.180075 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":54,"threads_started":1}
I20260925 22:07:37.181000 18269 tablet_service.cc:1467] Processing CreateTablet for tablet 675b193f8c284cbe8aca9710cca42356 (DEFAULT_TABLE table=test-table [id=a925d0dc469241a59294392e8524c251]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:07:37.181354 18269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 675b193f8c284cbe8aca9710cca42356. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:07:37.182358 18370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:37.174592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:41819 (local address 127.0.0.1:59250)
0925 22:07:37.176093 (+ 1501us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:37.176100 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:37.176182 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:37.176643 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:37.176644 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:37.176646 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:07:37.176684 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:37.176686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:37.180492 (+ 3806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:37.180492 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:37.181380 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:37.181383 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:37.182089 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:37.182090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:37.182144 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:37.182147 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:07:37.182248 (+ 101us) client_negotiation.cc:240] Negotiation successful
0925 22:07:37.182288 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1489,"thread_start_us":82,"threads_started":1}
I20260925 22:07:37.182385 18374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:37.175213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59250 (local address 127.5.0.65:41819)
0925 22:07:37.175344 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:37.175348 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:37.176154 (+ 806us) server_negotiation.cc:407] Connection header received
0925 22:07:37.176303 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:37.176305 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:37.176320 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:37.176370 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:07:37.179333 (+ 2963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:37.180394 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:37.181491 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:37.181976 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:37.182011 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:37.182042 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:37.182267 (+ 225us) server_negotiation.cc:299] Negotiation successful
0925 22:07:37.182310 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":84,"threads_started":1}
I20260925 22:07:37.183473 18008 tablet_service.cc:1467] Processing CreateTablet for tablet 675b193f8c284cbe8aca9710cca42356 (DEFAULT_TABLE table=test-table [id=a925d0dc469241a59294392e8524c251]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:07:37.183739 18372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:37.174896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56456 (local address 127.5.0.66:33067)
0925 22:07:37.175010 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:37.175015 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:37.176050 (+ 1035us) server_negotiation.cc:407] Connection header received
0925 22:07:37.176485 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:37.176487 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:37.176502 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:37.176549 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:07:37.176837 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:37.177917 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:37.182487 (+ 4570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:37.183271 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:37.183321 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:37.183369 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:37.183614 (+ 245us) server_negotiation.cc:299] Negotiation successful
0925 22:07:37.183664 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":65,"threads_started":1}
I20260925 22:07:37.183847 18008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 675b193f8c284cbe8aca9710cca42356. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:07:37.183945 18369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:37.174367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:33067 (local address 127.0.0.1:56456)
0925 22:07:37.175980 (+ 1613us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:37.175991 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:37.176225 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:37.176750 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:37.176751 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:37.176752 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:07:37.176773 (+ 21us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:37.176774 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:37.178130 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:37.178131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:37.179236 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:37.179238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:37.183452 (+ 4214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:37.183453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:37.183518 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:37.183522 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:07:37.183810 (+ 288us) client_negotiation.cc:240] Negotiation successful
0925 22:07:37.183856 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1596,"mutex_wait_us":2,"thread_start_us":91,"threads_started":1}
I20260925 22:07:37.185024 18375 tablet_bootstrap.cc:492] T 675b193f8c284cbe8aca9710cca42356 P c639c7d469cb4ab59c886ca28bdc1624: Bootstrap starting.
I20260925 22:07:37.185676 18375 tablet_bootstrap.cc:654] T 675b193f8c284cbe8aca9710cca42356 P c639c7d469cb4ab59c886ca28bdc1624: Neither blocks nor log segments found. Creating new log.
I20260925 22:07:37.185983 18375 log.cc:824] T 675b193f8c284cbe8aca9710cca42356 P c639c7d469cb4ab59c886ca28bdc1624: Log is configured to *not* fsync() on all Append() calls
I20260925 22:07:37.186869 18377 tablet_bootstrap.cc:492] T 675b193f8c284cbe8aca9710cca42356 P 4761cd7b68cf4c83b96e897c386d4b5a: Bootstrap starting.
I20260925 22:07:37.187493 18377 tablet_bootstrap.cc:654] T 675b193f8c284cbe8aca9710cca42356 P 4761cd7b68cf4c83b96e897c386d4b5a: Neither blocks nor log segments found. Creating new log.
I20260925 22:07:37.187770 18377 log.cc:824] T 675b193f8c284cbe8aca9710cca42356 P 4761cd7b68cf4c83b96e897c386d4b5a: Log is configured to *not* fsync() on all Append() calls
I20260925 22:07:37.187831 18142 tablet_service.cc:1467] Processing CreateTablet for tablet 675b193f8c284cbe8aca9710cca42356 (DEFAULT_TABLE table=test-table [id=a925d0dc469241a59294392e8524c251]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:07:37.188521 18142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 675b193f8c284cbe8aca9710cca42356. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:07:37.190951 18379 tablet_bootstrap.cc:492] T 675b193f8c284cbe8aca9710cca42356 P dff6386c354f4c4cb813d458441a2d1b: Bootstrap starting.
I20260925 22:07:37.191699 18379 tablet_bootstrap.cc:654] T 675b193f8c284cbe8aca9710cca42356 P dff6386c354f4c4cb813d458441a2d1b: Neither blocks nor log segments found. Creating new log.
I20260925 22:07:37.192075 18377 tablet_bootstrap.cc:492] T 675b193f8c284cbe8aca9710cca42356 P 4761cd7b68cf4c83b96e897c386d4b5a: No bootstrap required, opened a new log
I20260925 22:07:37.192152 18377 ts_tablet_manager.cc:1397] T 675b193f8c284cbe8aca9710cca42356 P 4761cd7b68cf4c83b96e897c386d4b5a: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260925 22:07:37.192093 18379 log.cc:824] T 675b193f8c284cbe8aca9710cca42356 P dff6386c354f4c4cb813d458441a2d1b: Log is configured to *not* fsync() on all Append() calls
I20260925 22:07:37.193998 18377 raft_consensus.cc:348] T 675b193f8c284cbe8aca9710cca42356 P 4761cd7b68cf4c83b96e897c386d4b5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dff6386c354f4c4cb813d458441a2d1b" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33067 } } peers { permanent_uuid: "c639c7d469cb4ab59c886ca28bdc1624" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 34539 } } peers { permanent_uuid: "4761cd7b68cf4c83b96e897c386d4b5a" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 41819 } }
I20260925 22:07:37.194161 18377 raft_consensus.cc:374] T 675b193f8c284cbe8aca9710cca42356 P 4761cd7b68cf4c83b96e897c386d4b5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:07:37.194207 18377 raft_consensus.cc:729] T 675b193f8c284cbe8aca9710cca42356 P 4761cd7b68cf4c83b96e897c386d4b5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4761cd7b68cf4c83b96e897c386d4b5a, State: Initialized, Role: FOLLOWER
I20260925 22:07:37.194341 18377 consensus_queue.cc:260] T 675b193f8c284cbe8aca9710cca42356 P 4761cd7b68cf4c83b96e897c386d4b5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dff6386c354f4c4cb813d458441a2d1b" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33067 } } peers { permanent_uuid: "c639c7d469cb4ab59c886ca28bdc1624" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 34539 } } peers { permanent_uuid: "4761cd7b68cf4c83b96e897c386d4b5a" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 41819 } }
I20260925 22:07:37.194612 18377 ts_tablet_manager.cc:1428] T 675b193f8c284cbe8aca9710cca42356 P 4761cd7b68cf4c83b96e897c386d4b5a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 22:07:37.194742 18074 heartbeater.cc:503] Master 127.5.0.126:36985 was elected leader, sending a full tablet report...
I20260925 22:07:37.194942 18379 tablet_bootstrap.cc:492] T 675b193f8c284cbe8aca9710cca42356 P dff6386c354f4c4cb813d458441a2d1b: No bootstrap required, opened a new log
I20260925 22:07:37.195000 18379 ts_tablet_manager.cc:1397] T 675b193f8c284cbe8aca9710cca42356 P dff6386c354f4c4cb813d458441a2d1b: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260925 22:07:37.196552 18375 tablet_bootstrap.cc:492] T 675b193f8c284cbe8aca9710cca42356 P c639c7d469cb4ab59c886ca28bdc1624: No bootstrap required, opened a new log
I20260925 22:07:37.196617 18375 ts_tablet_manager.cc:1397] T 675b193f8c284cbe8aca9710cca42356 P c639c7d469cb4ab59c886ca28bdc1624: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20260925 22:07:37.197520 18379 raft_consensus.cc:348] T 675b193f8c284cbe8aca9710cca42356 P dff6386c354f4c4cb813d458441a2d1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dff6386c354f4c4cb813d458441a2d1b" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33067 } } peers { permanent_uuid: "c639c7d469cb4ab59c886ca28bdc1624" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 34539 } } peers { permanent_uuid: "4761cd7b68cf4c83b96e897c386d4b5a" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 41819 } }
I20260925 22:07:37.197664 18379 raft_consensus.cc:374] T 675b193f8c284cbe8aca9710cca42356 P dff6386c354f4c4cb813d458441a2d1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:07:37.197696 18379 raft_consensus.cc:729] T 675b193f8c284cbe8aca9710cca42356 P dff6386c354f4c4cb813d458441a2d1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dff6386c354f4c4cb813d458441a2d1b, State: Initialized, Role: FOLLOWER
I20260925 22:07:37.197940 18379 consensus_queue.cc:260] T 675b193f8c284cbe8aca9710cca42356 P dff6386c354f4c4cb813d458441a2d1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dff6386c354f4c4cb813d458441a2d1b" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33067 } } peers { permanent_uuid: "c639c7d469cb4ab59c886ca28bdc1624" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 34539 } } peers { permanent_uuid: "4761cd7b68cf4c83b96e897c386d4b5a" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 41819 } }
I20260925 22:07:37.198268 18379 ts_tablet_manager.cc:1428] T 675b193f8c284cbe8aca9710cca42356 P dff6386c354f4c4cb813d458441a2d1b: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260925 22:07:37.198422 18208 heartbeater.cc:503] Master 127.5.0.126:36985 was elected leader, sending a full tablet report...
I20260925 22:07:37.199049 18375 raft_consensus.cc:348] T 675b193f8c284cbe8aca9710cca42356 P c639c7d469cb4ab59c886ca28bdc1624 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dff6386c354f4c4cb813d458441a2d1b" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33067 } } peers { permanent_uuid: "c639c7d469cb4ab59c886ca28bdc1624" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 34539 } } peers { permanent_uuid: "4761cd7b68cf4c83b96e897c386d4b5a" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 41819 } }
I20260925 22:07:37.199216 18375 raft_consensus.cc:374] T 675b193f8c284cbe8aca9710cca42356 P c639c7d469cb4ab59c886ca28bdc1624 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:07:37.199261 18375 raft_consensus.cc:729] T 675b193f8c284cbe8aca9710cca42356 P c639c7d469cb4ab59c886ca28bdc1624 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c639c7d469cb4ab59c886ca28bdc1624, State: Initialized, Role: FOLLOWER
I20260925 22:07:37.199522 18375 consensus_queue.cc:260] T 675b193f8c284cbe8aca9710cca42356 P c639c7d469cb4ab59c886ca28bdc1624 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dff6386c354f4c4cb813d458441a2d1b" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33067 } } peers { permanent_uuid: "c639c7d469cb4ab59c886ca28bdc1624" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 34539 } } peers { permanent_uuid: "4761cd7b68cf4c83b96e897c386d4b5a" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 41819 } }
I20260925 22:07:37.199795 18375 ts_tablet_manager.cc:1428] T 675b193f8c284cbe8aca9710cca42356 P c639c7d469cb4ab59c886ca28bdc1624: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 22:07:37.199939 18342 heartbeater.cc:503] Master 127.5.0.126:36985 was elected leader, sending a full tablet report...
I20260925 22:07:37.208632 18383 raft_consensus.cc:482] T 675b193f8c284cbe8aca9710cca42356 P c639c7d469cb4ab59c886ca28bdc1624 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:07:37.208739 18383 raft_consensus.cc:504] T 675b193f8c284cbe8aca9710cca42356 P c639c7d469cb4ab59c886ca28bdc1624 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dff6386c354f4c4cb813d458441a2d1b" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33067 } } peers { permanent_uuid: "c639c7d469cb4ab59c886ca28bdc1624" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 34539 } } peers { permanent_uuid: "4761cd7b68cf4c83b96e897c386d4b5a" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 41819 } }
I20260925 22:07:37.209292 18383 leader_election.cc:288] T 675b193f8c284cbe8aca9710cca42356 P c639c7d469cb4ab59c886ca28bdc1624 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dff6386c354f4c4cb813d458441a2d1b (127.5.0.66:33067), 4761cd7b68cf4c83b96e897c386d4b5a (127.5.0.65:41819)
W20260925 22:07:37.211838 18209 tablet.cc:2369] T 675b193f8c284cbe8aca9710cca42356 P dff6386c354f4c4cb813d458441a2d1b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:07:37.213281 18372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:37.209392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:50233 (local address 127.5.0.66:33067)
0925 22:07:37.209435 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:37.209437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:37.209713 (+ 276us) server_negotiation.cc:407] Connection header received
0925 22:07:37.209807 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:37.209808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:37.209820 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:37.209847 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:07:37.210030 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:37.210709 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:37.212320 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:37.212872 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:37.212922 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:37.212931 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:37.213192 (+ 261us) server_negotiation.cc:299] Negotiation successful
0925 22:07:37.213229 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:07:37.213281 18374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:37.209413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:46021 (local address 127.5.0.65:41819)
0925 22:07:37.209470 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:37.209473 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:37.209566 (+ 93us) server_negotiation.cc:407] Connection header received
0925 22:07:37.209642 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:37.209643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:37.209655 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:37.209708 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:07:37.209922 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:37.210678 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:37.212320 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:37.212890 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:37.212926 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:37.212934 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:37.213176 (+ 242us) server_negotiation.cc:299] Negotiation successful
0925 22:07:37.213215 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 22:07:37.213630 18384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:37.209177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:33067 (local address 127.5.0.67:50233)
0925 22:07:37.209665 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:37.209673 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:37.209736 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:37.209907 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:37.209908 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:37.209910 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:07:37.209961 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:37.209962 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:37.210788 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:37.210789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:37.212216 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:37.212219 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:37.213014 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:37.213015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:37.213092 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:37.213096 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:07:37.213493 (+ 397us) client_negotiation.cc:240] Negotiation successful
0925 22:07:37.213534 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"mutex_wait_us":118,"thread_start_us":76,"threads_started":1}
I20260925 22:07:37.214061 18162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "675b193f8c284cbe8aca9710cca42356" candidate_uuid: "c639c7d469cb4ab59c886ca28bdc1624" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dff6386c354f4c4cb813d458441a2d1b" is_pre_election: true
I20260925 22:07:37.214221 18162 raft_consensus.cc:2454] T 675b193f8c284cbe8aca9710cca42356 P dff6386c354f4c4cb813d458441a2d1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c639c7d469cb4ab59c886ca28bdc1624 in term 0.
I20260925 22:07:37.214403 18385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:37.209177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:41819 (local address 127.5.0.67:46021)
0925 22:07:37.209510 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:37.209521 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:37.209581 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:37.209758 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:37.209759 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:37.209761 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:07:37.209841 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:37.209843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:37.210777 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:37.210778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:37.212216 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:37.212219 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:37.213014 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:37.213015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:37.213092 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:37.213096 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:07:37.213581 (+ 485us) client_negotiation.cc:240] Negotiation successful
0925 22:07:37.213616 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":45,"thread_start_us":87,"threads_started":1}
I20260925 22:07:37.214432 18230 leader_election.cc:302] T 675b193f8c284cbe8aca9710cca42356 P c639c7d469cb4ab59c886ca28bdc1624 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c639c7d469cb4ab59c886ca28bdc1624, dff6386c354f4c4cb813d458441a2d1b; no voters:
I20260925 22:07:37.214690 18383 raft_consensus.cc:2781] T 675b193f8c284cbe8aca9710cca42356 P c639c7d469cb4ab59c886ca28bdc1624 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:07:37.214684 18028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "675b193f8c284cbe8aca9710cca42356" candidate_uuid: "c639c7d469cb4ab59c886ca28bdc1624" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4761cd7b68cf4c83b96e897c386d4b5a" is_pre_election: true
I20260925 22:07:37.214761 18383 raft_consensus.cc:482] T 675b193f8c284cbe8aca9710cca42356 P c639c7d469cb4ab59c886ca28bdc1624 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:07:37.214788 18383 raft_consensus.cc:3037] T 675b193f8c284cbe8aca9710cca42356 P c639c7d469cb4ab59c886ca28bdc1624 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:07:37.214823 18028 raft_consensus.cc:2454] T 675b193f8c284cbe8aca9710cca42356 P 4761cd7b68cf4c83b96e897c386d4b5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c639c7d469cb4ab59c886ca28bdc1624 in term 0.
I20260925 22:07:37.215615 18383 raft_consensus.cc:504] T 675b193f8c284cbe8aca9710cca42356 P c639c7d469cb4ab59c886ca28bdc1624 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dff6386c354f4c4cb813d458441a2d1b" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33067 } } peers { permanent_uuid: "c639c7d469cb4ab59c886ca28bdc1624" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 34539 } } peers { permanent_uuid: "4761cd7b68cf4c83b96e897c386d4b5a" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 41819 } }
I20260925 22:07:37.215756 18383 leader_election.cc:288] T 675b193f8c284cbe8aca9710cca42356 P c639c7d469cb4ab59c886ca28bdc1624 [CANDIDATE]: Term 1 election: Requested vote from peers dff6386c354f4c4cb813d458441a2d1b (127.5.0.66:33067), 4761cd7b68cf4c83b96e897c386d4b5a (127.5.0.65:41819)
I20260925 22:07:37.215934 18028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "675b193f8c284cbe8aca9710cca42356" candidate_uuid: "c639c7d469cb4ab59c886ca28bdc1624" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4761cd7b68cf4c83b96e897c386d4b5a"
I20260925 22:07:37.215969 18162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "675b193f8c284cbe8aca9710cca42356" candidate_uuid: "c639c7d469cb4ab59c886ca28bdc1624" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dff6386c354f4c4cb813d458441a2d1b"
I20260925 22:07:37.216005 18028 raft_consensus.cc:3037] T 675b193f8c284cbe8aca9710cca42356 P 4761cd7b68cf4c83b96e897c386d4b5a [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:07:37.216034 18162 raft_consensus.cc:3037] T 675b193f8c284cbe8aca9710cca42356 P dff6386c354f4c4cb813d458441a2d1b [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:07:37.216600 18162 raft_consensus.cc:2454] T 675b193f8c284cbe8aca9710cca42356 P dff6386c354f4c4cb813d458441a2d1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c639c7d469cb4ab59c886ca28bdc1624 in term 1.
I20260925 22:07:37.216781 18230 leader_election.cc:302] T 675b193f8c284cbe8aca9710cca42356 P c639c7d469cb4ab59c886ca28bdc1624 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c639c7d469cb4ab59c886ca28bdc1624, dff6386c354f4c4cb813d458441a2d1b; no voters:
I20260925 22:07:37.216840 18028 raft_consensus.cc:2454] T 675b193f8c284cbe8aca9710cca42356 P 4761cd7b68cf4c83b96e897c386d4b5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c639c7d469cb4ab59c886ca28bdc1624 in term 1.
I20260925 22:07:37.216861 18383 raft_consensus.cc:2781] T 675b193f8c284cbe8aca9710cca42356 P c639c7d469cb4ab59c886ca28bdc1624 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:07:37.216993 18383 raft_consensus.cc:686] T 675b193f8c284cbe8aca9710cca42356 P c639c7d469cb4ab59c886ca28bdc1624 [term 1 LEADER]: Becoming Leader. State: Replica: c639c7d469cb4ab59c886ca28bdc1624, State: Running, Role: LEADER
I20260925 22:07:37.217080 18383 consensus_queue.cc:237] T 675b193f8c284cbe8aca9710cca42356 P c639c7d469cb4ab59c886ca28bdc1624 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dff6386c354f4c4cb813d458441a2d1b" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33067 } } peers { permanent_uuid: "c639c7d469cb4ab59c886ca28bdc1624" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 34539 } } peers { permanent_uuid: "4761cd7b68cf4c83b96e897c386d4b5a" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 41819 } }
I20260925 22:07:37.217803 17875 catalog_manager.cc:5495] T 675b193f8c284cbe8aca9710cca42356 P c639c7d469cb4ab59c886ca28bdc1624 reported cstate change: term changed from 0 to 1, leader changed from <none> to c639c7d469cb4ab59c886ca28bdc1624 (127.5.0.67). New cstate: current_term: 1 leader_uuid: "c639c7d469cb4ab59c886ca28bdc1624" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dff6386c354f4c4cb813d458441a2d1b" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c639c7d469cb4ab59c886ca28bdc1624" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 34539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4761cd7b68cf4c83b96e897c386d4b5a" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 41819 } health_report { overall_health: UNKNOWN } } }
I20260925 22:07:37.229010 18373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:37.226542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45592 (local address 127.5.0.67:34539)
0925 22:07:37.226582 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:37.226584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:37.226592 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:07:37.226679 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:37.226679 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:37.226697 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:37.226782 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:07:37.226885 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:37.227348 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:37.228185 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:37.228388 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:37.228473 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:37.228672 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:07:37.228780 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:07:37.228865 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:37.228905 (+ 40us) server_negotiation.cc:299] Negotiation successful
0925 22:07:37.228924 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 22:07:37.230779 18162 raft_consensus.cc:1261] T 675b193f8c284cbe8aca9710cca42356 P dff6386c354f4c4cb813d458441a2d1b [term 1 FOLLOWER]: Refusing update from remote peer c639c7d469cb4ab59c886ca28bdc1624: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:07:37.230779 18028 raft_consensus.cc:1261] T 675b193f8c284cbe8aca9710cca42356 P 4761cd7b68cf4c83b96e897c386d4b5a [term 1 FOLLOWER]: Refusing update from remote peer c639c7d469cb4ab59c886ca28bdc1624: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:07:37.231072 18383 consensus_queue.cc:1035] T 675b193f8c284cbe8aca9710cca42356 P c639c7d469cb4ab59c886ca28bdc1624 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4761cd7b68cf4c83b96e897c386d4b5a" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 41819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:07:37.231230 18383 consensus_queue.cc:1035] T 675b193f8c284cbe8aca9710cca42356 P c639c7d469cb4ab59c886ca28bdc1624 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dff6386c354f4c4cb813d458441a2d1b" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 33067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:07:37.234802 18389 mvcc.cc:205] Tried to move back new op lower bound from 7333372138414907392 to 7333372138361675776. Current Snapshot: MvccSnapshot[applied={T|T < 7333372138414907392}]
I20260925 22:07:37.235312 18390 mvcc.cc:205] Tried to move back new op lower bound from 7333372138414907392 to 7333372138361675776. Current Snapshot: MvccSnapshot[applied={T|T < 7333372138414907392}]
I20260925 22:07:37.235311 18391 mvcc.cc:205] Tried to move back new op lower bound from 7333372138414907392 to 7333372138361675776. Current Snapshot: MvccSnapshot[applied={T|T < 7333372138414907392}]
I20260925 22:07:37.237720 17875 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:07:37.237835 17875 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:07:37.238523 17875 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5c53f46f65344a487f29d36b547e6cd: Sending DeleteTablet for 3 replicas of tablet 675b193f8c284cbe8aca9710cca42356
I20260925 22:07:37.238874 18269 tablet_service.cc:1514] Processing DeleteTablet for tablet 675b193f8c284cbe8aca9710cca42356 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:07:37 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:45588
I20260925 22:07:37.238881 18142 tablet_service.cc:1514] Processing DeleteTablet for tablet 675b193f8c284cbe8aca9710cca42356 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:07:37 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:56456
I20260925 22:07:37.238940 18008 tablet_service.cc:1514] Processing DeleteTablet for tablet 675b193f8c284cbe8aca9710cca42356 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:07:37 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:59250
I20260925 22:07:37.239382 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 17914
I20260925 22:07:37.244030 18399 tablet_replica.cc:330] T 675b193f8c284cbe8aca9710cca42356 P dff6386c354f4c4cb813d458441a2d1b: stopping tablet replica
I20260925 22:07:37.244189 18399 raft_consensus.cc:2229] T 675b193f8c284cbe8aca9710cca42356 P dff6386c354f4c4cb813d458441a2d1b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:07:37.244379 18399 raft_consensus.cc:2258] T 675b193f8c284cbe8aca9710cca42356 P dff6386c354f4c4cb813d458441a2d1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:07:37.244612 18398 tablet_replica.cc:330] T 675b193f8c284cbe8aca9710cca42356 P c639c7d469cb4ab59c886ca28bdc1624: stopping tablet replica
I20260925 22:07:37.248744 18399 ts_tablet_manager.cc:1905] T 675b193f8c284cbe8aca9710cca42356 P dff6386c354f4c4cb813d458441a2d1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:07:37.250492 18398 raft_consensus.cc:2229] T 675b193f8c284cbe8aca9710cca42356 P c639c7d469cb4ab59c886ca28bdc1624 [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:07:37.250703 18398 raft_consensus.cc:2258] T 675b193f8c284cbe8aca9710cca42356 P c639c7d469cb4ab59c886ca28bdc1624 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 22:07:37.250993 17862 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.0.65:41819 (error 108)
I20260925 22:07:37.251230 18398 ts_tablet_manager.cc:1905] T 675b193f8c284cbe8aca9710cca42356 P c639c7d469cb4ab59c886ca28bdc1624: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:07:37.252013 18399 ts_tablet_manager.cc:1918] T 675b193f8c284cbe8aca9710cca42356 P dff6386c354f4c4cb813d458441a2d1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:07:37.252038 18369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:37.251920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:41819 (local address 127.0.0.1:59258)
0925 22:07:37.251960 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:37.251997 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.65:41819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260925 22:07:37.252049 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 18078
I20260925 22:07:37.252089 18399 log.cc:1196] T 675b193f8c284cbe8aca9710cca42356 P dff6386c354f4c4cb813d458441a2d1b: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-1/wal/wals/675b193f8c284cbe8aca9710cca42356
W20260925 22:07:37.252621 17862 catalog_manager.cc:4584] TS 4761cd7b68cf4c83b96e897c386d4b5a (127.5.0.65:41819): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 675b193f8c284cbe8aca9710cca42356: Network error: Client connection negotiation failed: client connection to 127.5.0.65:41819: connect: Connection refused (error 111)
I20260925 22:07:37.253655 18398 ts_tablet_manager.cc:1918] T 675b193f8c284cbe8aca9710cca42356 P c639c7d469cb4ab59c886ca28bdc1624: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:07:37.253712 18398 log.cc:1196] T 675b193f8c284cbe8aca9710cca42356 P c639c7d469cb4ab59c886ca28bdc1624: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373908987421-5121-0/minicluster-data/ts-2/wal/wals/675b193f8c284cbe8aca9710cca42356
I20260925 22:07:37.253966 18398 ts_tablet_manager.cc:1939] T 675b193f8c284cbe8aca9710cca42356 P c639c7d469cb4ab59c886ca28bdc1624: Deleting consensus metadata
I20260925 22:07:37.254350 17862 catalog_manager.cc:4841] TS c639c7d469cb4ab59c886ca28bdc1624 (127.5.0.67:34539): tablet 675b193f8c284cbe8aca9710cca42356 (table test-table [id=a925d0dc469241a59294392e8524c251]) successfully deleted
I20260925 22:07:37.259866 18369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:37.259736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:33067 (local address 127.0.0.1:56470)
0925 22:07:37.259801 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:37.259830 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.66:33067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260925 22:07:37.263108 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 18211
I20260925 22:07:37.268203 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 17843
I20260925 22:07:37.274791 5121 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260925 22:07:37.609623 5121 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260925 22:07:37.609726 5121 mini_ranger.cc:66] Stopping Ranger...
I20260925 22:07:37.954385 5121 mini_ranger.cc:68] Stopped Ranger
2026-09-25 22:07:37.954 UTC [16910] LOG: received smart shutdown request
2026-09-25 22:07:37.957 UTC [16910] LOG: background worker "logical replication launcher" (PID 16919) exited with exit code 1
2026-09-25 22:07:37.958 UTC [16914] LOG: shutting down
2026-09-25 22:07:38.128 UTC [16910] LOG: database system is shut down
2026-09-25T22:07:38Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (46498 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:07:38 dist-test-slave-7mqr krb5kdc[18410](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:07:38 dist-test-slave-7mqr krb5kdc[18410](info): set up 2 sockets
Sep 25 22:07:38 dist-test-slave-7mqr krb5kdc[18410](info): commencing operation
krb5kdc: starting...
W20260925 22:07:39.271751 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:07:39 dist-test-slave-7mqr krb5kdc[18410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:07:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:07:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.0.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
I20260925 22:07:39.386405 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:43827
--webserver_interface=127.5.0.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:37581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:43827,127.5.0.125:46391,127.5.0.124:43427
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:07:39.467576 18423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:07:39.467820 18423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:07:39.467859 18423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:07:39.469030 18423 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:07:39.469099 18423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:07:39.469123 18423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:07:39.469152 18423 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:07:39.469184 18423 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:07:39.470692 18423 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:37581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:43827,127.5.0.125:46391,127.5.0.124:43427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:43827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:07:39.470974 18423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:07:39.471199 18423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:07:39.475937 18431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:39.475909 18429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:39.475958 18428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:07:39.476272 18423 server_base.cc:1034] running on GCE node
I20260925 22:07:39.476557 18423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:07:39.476850 18423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:07:39.478017 18423 hybrid_clock.cc:648] HybridClock initialized: now 1790374059478017 us; error 42 us; skew 500 ppm
Sep 25 22:07:39 dist-test-slave-7mqr krb5kdc[18410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374059, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:07:39.480544 18423 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:07:39.480983 18423 webserver.cc:460] Webserver started at http://127.5.0.126:39007/ using document root <none> and password file <none>
I20260925 22:07:39.481232 18423 fs_manager.cc:359] Metadata directory not provided
I20260925 22:07:39.481302 18423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:07:39.481449 18423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:07:39.482507 18423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "d1bb831817514aa3b3ce02af42da76a2"
format_stamp: "Formatted at 2026-09-25 22:07:39 on dist-test-slave-7mqr"
server_key: "e80fa9c9fe700f946830ea3fec9a299e"
server_key_iv: "0376479c44c7dd221e1104452755159e"
server_key_version: "encryptionkey@0"
I20260925 22:07:39.482919 18423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "d1bb831817514aa3b3ce02af42da76a2"
format_stamp: "Formatted at 2026-09-25 22:07:39 on dist-test-slave-7mqr"
server_key: "e80fa9c9fe700f946830ea3fec9a299e"
server_key_iv: "0376479c44c7dd221e1104452755159e"
server_key_version: "encryptionkey@0"
I20260925 22:07:39.484289 18423 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:07:39.485023 18438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:39.485198 18423 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:07:39.485306 18423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "d1bb831817514aa3b3ce02af42da76a2"
format_stamp: "Formatted at 2026-09-25 22:07:39 on dist-test-slave-7mqr"
server_key: "e80fa9c9fe700f946830ea3fec9a299e"
server_key_iv: "0376479c44c7dd221e1104452755159e"
server_key_version: "encryptionkey@0"
I20260925 22:07:39.485381 18423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:07:39.500957 18423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:07:39.502825 18423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:07:39.502995 18423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:07:39.506982 18423 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:43827
I20260925 22:07:39.507018 18490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:43827 every 8 connection(s)
I20260925 22:07:39.507392 18423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:07:39.508291 18491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:07:39.509806 18491 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43827 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:07:39.511044 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 18423
I20260925 22:07:39.511168 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:07:39.511361 5121 external_mini_cluster.cc:1428] Setting key c22583e3d45a25be421ac015c6b003b4
Sep 25 22:07:39 dist-test-slave-7mqr krb5kdc[18410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374059, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.0.125@KRBTEST.COM; defaulting to no policy
I20260925 22:07:39.519946 18493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.510463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:43827 (local address 127.0.0.1:54400)
0925 22:07:39.510617 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:39.510652 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:39.511120 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:39.511430 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:39.511432 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:39.511731 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:07:39.511924 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:39.511931 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.512965 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.512966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:39.513655 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:39.513657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.514344 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.514345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:39.514413 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:39.514939 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:07:39.514947 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:07:39.516414 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:07:39.518472 (+ 2058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:39.518478 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:39.518480 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:39.518664 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:39.518935 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:39.518936 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:39.518936 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:39.519008 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:39.519425 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:07:39.519429 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:07:39.519482 (+ 53us) client_negotiation.cc:769] Sending connection context
0925 22:07:39.519592 (+ 110us) client_negotiation.cc:240] Negotiation successful
0925 22:07:39.519737 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":61,"threads_started":1}
I20260925 22:07:39.519961 18494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.510605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54400 (local address 127.5.0.126:43827)
0925 22:07:39.510743 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:39.510747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:39.510774 (+ 27us) server_negotiation.cc:407] Connection header received
0925 22:07:39.511306 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:39.511308 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:39.511326 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:39.511424 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:39.512039 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.512867 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.513760 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.514254 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.514362 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:39.516502 (+ 2140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:39.516511 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:39.516511 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:39.516519 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:39.518387 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:39.518750 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:39.518753 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:39.518753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:39.518859 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:39.519076 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:39.519077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:39.519078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:39.519355 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:39.519443 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:39.519645 (+ 202us) server_negotiation.cc:299] Negotiation successful
0925 22:07:39.519820 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":98,"thread_start_us":72,"threads_started":1}
I20260925 22:07:39.520738 18491 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.0.125" port: 46391 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:07:39.521060 18493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.520944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:46391 (local address 127.0.0.1:41738)
0925 22:07:39.520992 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:39.521033 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.125:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260925 22:07:39.521224 18440 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.0.125:46391: connect: Connection refused (error 111)
I20260925 22:07:39.521678 18493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.521568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:46391 (local address 127.0.0.1:41742)
0925 22:07:39.521598 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:39.521627 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.125:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260925 22:07:39.521803 18491 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.0.125:46391: Network error: Client connection negotiation failed: client connection to 127.5.0.125:46391: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.125.keytab.
Entry for principal kudu/127.5.0.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.125.keytab.
I20260925 22:07:39.536859 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.125:46391
--webserver_interface=127.5.0.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.125.keytab
--principal=kudu/127.5.0.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:37581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:43827,127.5.0.125:46391,127.5.0.124:43427
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:07:39.571007 18491 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.0.125" port: 46391 } attempt: 1
I20260925 22:07:39.571539 18493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.571380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:46391 (local address 127.0.0.1:41744)
0925 22:07:39.571445 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:39.571497 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.125:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260925 22:07:39.571993 18493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.571845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:46391 (local address 127.0.0.1:41756)
0925 22:07:39.571923 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:39.571958 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.125:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260925 22:07:39.572122 18491 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.0.125:46391: Network error: Client connection negotiation failed: client connection to 127.5.0.125:46391: connect: Connection refused (error 111)
W20260925 22:07:39.616983 18498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:07:39.617185 18498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:07:39.617208 18498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:07:39.618330 18498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:07:39.618371 18498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:07:39.618386 18498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:07:39.618397 18498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:07:39.618448 18498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:07:39.619933 18498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:37581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:43827,127.5.0.125:46391,127.5.0.124:43427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.125.keytab
--principal=kudu/127.5.0.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.125:46391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.0.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:07:39.620187 18498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:07:39.620404 18498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:07:39.625092 18506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:39.625147 18503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:39.625195 18504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:07:39.625356 18498 server_base.cc:1034] running on GCE node
I20260925 22:07:39.625612 18498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:07:39.625910 18498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:07:39.627053 18498 hybrid_clock.cc:648] HybridClock initialized: now 1790374059627035 us; error 50 us; skew 500 ppm
Sep 25 22:07:39 dist-test-slave-7mqr krb5kdc[18410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374059, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:07:39.629616 18498 init.cc:377] Logged in from keytab as kudu/127.5.0.125@KRBTEST.COM (short username kudu)
I20260925 22:07:39.630056 18498 webserver.cc:460] Webserver started at http://127.5.0.125:38045/ using document root <none> and password file <none>
I20260925 22:07:39.630294 18498 fs_manager.cc:359] Metadata directory not provided
I20260925 22:07:39.630362 18498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:07:39.630513 18498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:07:39.631587 18498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-1/data/instance:
uuid: "65ab35a533464e908a727861c0eb005a"
format_stamp: "Formatted at 2026-09-25 22:07:39 on dist-test-slave-7mqr"
server_key: "dec9d82a15c96a59eb960c1f19385093"
server_key_iv: "2ddaf0e4f733ff3a642fc10193ffdfd7"
server_key_version: "encryptionkey@0"
I20260925 22:07:39.632001 18498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-1/wal/instance:
uuid: "65ab35a533464e908a727861c0eb005a"
format_stamp: "Formatted at 2026-09-25 22:07:39 on dist-test-slave-7mqr"
server_key: "dec9d82a15c96a59eb960c1f19385093"
server_key_iv: "2ddaf0e4f733ff3a642fc10193ffdfd7"
server_key_version: "encryptionkey@0"
I20260925 22:07:39.633292 18498 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:07:39.633996 18513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:39.634157 18498 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:07:39.634254 18498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-1/wal
uuid: "65ab35a533464e908a727861c0eb005a"
format_stamp: "Formatted at 2026-09-25 22:07:39 on dist-test-slave-7mqr"
server_key: "dec9d82a15c96a59eb960c1f19385093"
server_key_iv: "2ddaf0e4f733ff3a642fc10193ffdfd7"
server_key_version: "encryptionkey@0"
I20260925 22:07:39.634338 18498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:07:39.640295 18491 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.0.125" port: 46391 } attempt: 2
I20260925 22:07:39.640684 18493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.640565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:46391 (local address 127.0.0.1:41766)
0925 22:07:39.640623 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:39.640658 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.125:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 22:07:39.640996 18493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.640914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:46391 (local address 127.0.0.1:41778)
0925 22:07:39.640956 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:39.640980 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.125:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260925 22:07:39.641088 18491 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.0.125:46391: Network error: Client connection negotiation failed: client connection to 127.5.0.125:46391: connect: Connection refused (error 111)
I20260925 22:07:39.643741 18498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:07:39.645469 18498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:07:39.645637 18498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:07:39.649416 18498 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.125:46391
I20260925 22:07:39.649447 18565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.125:46391 every 8 connection(s)
I20260925 22:07:39.649781 18498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-1/data/info.pb
I20260925 22:07:39.650576 18566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:07:39.651153 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 18498
I20260925 22:07:39.651269 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-1/wal/instance
I20260925 22:07:39.651482 5121 external_mini_cluster.cc:1428] Setting key f4e3f2003fe34073c1bc263533127ab9
I20260925 22:07:39.652875 18566 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43827 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.5.0.124@KRBTEST.COM; defaulting to no policy
Sep 25 22:07:39 dist-test-slave-7mqr krb5kdc[18410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374059, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.125@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.124@KRBTEST.COM" created.
I20260925 22:07:39.662364 18569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.653706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:43827 (local address 127.0.0.1:54414)
0925 22:07:39.653869 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:39.653903 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:39.654372 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:39.654651 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:39.654654 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:39.654950 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:07:39.655165 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:39.655170 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.655843 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.655844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:39.656697 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:39.656700 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.657100 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.657101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:39.657220 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:39.657759 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:07:39.657767 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:07:39.659135 (+ 1368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:07:39.660924 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:39.660931 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:39.660933 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:39.661235 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:39.661458 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:39.661459 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:39.661459 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:39.661590 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:39.661897 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:07:39.661901 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:07:39.662008 (+ 107us) client_negotiation.cc:769] Sending connection context
0925 22:07:39.662088 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 22:07:39.662207 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":79,"threads_started":1}
I20260925 22:07:39.662369 18494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.653717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54414 (local address 127.5.0.126:43827)
0925 22:07:39.653771 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:39.653773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:39.653984 (+ 211us) server_negotiation.cc:407] Connection header received
0925 22:07:39.654488 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:39.654490 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:39.654502 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:39.654579 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:39.655260 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.655748 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.656802 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.657005 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.657078 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:39.659226 (+ 2148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:39.659227 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:39.659227 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:39.659234 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:39.660818 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:39.661316 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:39.661317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:39.661318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:39.661386 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:39.661676 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:39.661677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:39.661678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:39.661828 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:39.661903 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:39.662133 (+ 230us) server_negotiation.cc:299] Negotiation successful
0925 22:07:39.662226 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 22:07:39.662964 18566 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.0.125" port: 46391 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 22:07:39 dist-test-slave-7mqr krb5kdc[18410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374059, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.125@KRBTEST.COM for kudu/127.5.0.125@KRBTEST.COM
I20260925 22:07:39.670501 18569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.663241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:46391 (local address 127.0.0.1:41782)
0925 22:07:39.663296 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:39.663310 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:39.663356 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:39.663594 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:39.663595 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:39.663753 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:07:39.663811 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:39.663813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.664678 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.664679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:39.665545 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:39.665547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.665981 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.665983 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:39.666068 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:39.666406 (+ 338us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:07:39.666423 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:07:39.667703 (+ 1280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:07:39.669394 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:39.669395 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:39.669395 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:39.669553 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:39.669756 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:39.669756 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:39.669757 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:39.669808 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:39.670175 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:07:39.670176 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:07:39.670213 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 22:07:39.670278 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 22:07:39.670372 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 22:07:39.670589 18571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.663278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41782 (local address 127.5.0.125:46391)
0925 22:07:39.663448 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:39.663452 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:39.663470 (+ 18us) server_negotiation.cc:407] Connection header received
0925 22:07:39.663497 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:39.663499 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:39.663509 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:39.663575 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:39.663880 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.664595 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.665627 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.665903 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.665960 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:39.667788 (+ 1828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:39.667795 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:39.667796 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:39.667803 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:39.669301 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:39.669636 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:39.669639 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:39.669640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:39.669691 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:39.669906 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:39.669907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:39.669907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:39.670100 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:39.670196 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:39.670305 (+ 109us) server_negotiation.cc:299] Negotiation successful
0925 22:07:39.670439 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":141,"thread_start_us":116,"threads_started":1}
I20260925 22:07:39.671175 18566 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.0.124" port: 43427 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:07:39.671530 18569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.671428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:43427 (local address 127.0.0.1:39842)
0925 22:07:39.671466 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:39.671506 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.124:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260925 22:07:39.671658 18517 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.0.124:43427: connect: Connection refused (error 111)
I20260925 22:07:39.672109 18569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.672042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:43427 (local address 127.0.0.1:39846)
0925 22:07:39.672069 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:39.672093 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.124:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260925 22:07:39.672223 18566 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.0.124:43427: Network error: Client connection negotiation failed: client connection to 127.5.0.124:43427: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.124.keytab.
Entry for principal kudu/127.5.0.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.124.keytab.
I20260925 22:07:39.678580 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.124:43427
--webserver_interface=127.5.0.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.124.keytab
--principal=kudu/127.5.0.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:37581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:43827,127.5.0.125:46391,127.5.0.124:43427
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:07:39.721438 18566 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.0.124" port: 43427 } attempt: 1
I20260925 22:07:39.721935 18569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.721803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:43427 (local address 127.0.0.1:39858)
0925 22:07:39.721864 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:39.721904 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.124:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260925 22:07:39.722374 18569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.722273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:43427 (local address 127.0.0.1:39862)
0925 22:07:39.722322 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:39.722350 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.124:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260925 22:07:39.722470 18566 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.0.124:43427: Network error: Client connection negotiation failed: client connection to 127.5.0.124:43427: connect: Connection refused (error 111)
I20260925 22:07:39.732244 18491 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.0.125" port: 46391 } attempt: 3
Sep 25 22:07:39 dist-test-slave-7mqr krb5kdc[18410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374059, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for kudu/127.5.0.125@KRBTEST.COM
I20260925 22:07:39.739354 18493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.732621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:46391 (local address 127.0.0.1:41798)
0925 22:07:39.732661 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:39.732673 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:39.732747 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:39.732891 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:39.732892 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:39.733051 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:07:39.733123 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:39.733125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.733805 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.733806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:39.734316 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:39.734318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.734716 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.734717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:39.734797 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:39.735323 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:07:39.735327 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:07:39.736683 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:07:39.738441 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:39.738442 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:39.738442 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:39.738626 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:39.738819 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:39.738819 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:39.738819 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:39.738870 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:39.739130 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:07:39.739131 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:07:39.739168 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 22:07:39.739218 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 22:07:39.739289 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260925 22:07:39.739434 18571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.732680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41798 (local address 127.5.0.125:46391)
0925 22:07:39.732740 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:39.732741 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:39.732752 (+ 11us) server_negotiation.cc:407] Connection header received
0925 22:07:39.732817 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:39.732818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:39.732829 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:39.732885 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:39.733205 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.733722 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.734411 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.734631 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.734716 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:39.736784 (+ 2068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:39.736786 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:39.736786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:39.736794 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:39.738350 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:39.738692 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:39.738693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:39.738694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:39.738755 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:39.738927 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:39.738928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:39.738928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:39.739065 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:39.739137 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:39.739257 (+ 120us) server_negotiation.cc:299] Negotiation successful
0925 22:07:39.739338 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 22:07:39.739854 18491 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.0.124" port: 43427 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:07:39.740208 18493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.740111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:43427 (local address 127.0.0.1:39872)
0925 22:07:39.740160 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:39.740188 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.124:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260925 22:07:39.740676 18493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.740566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:43427 (local address 127.0.0.1:39888)
0925 22:07:39.740612 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:39.740650 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.124:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260925 22:07:39.740767 18491 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.0.124:43427: Network error: Client connection negotiation failed: client connection to 127.5.0.124:43427: connect: Connection refused (error 111)
W20260925 22:07:39.760540 18573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:07:39.760739 18573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:07:39.760761 18573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:07:39.761818 18573 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:07:39.761852 18573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:07:39.761866 18573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:07:39.761878 18573 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:07:39.761890 18573 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:07:39.763316 18573 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:37581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:43827,127.5.0.125:46391,127.5.0.124:43427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.124.keytab
--principal=kudu/127.5.0.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.124:43427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.0.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:07:39.763497 18573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:07:39.763679 18573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:07:39.768177 18579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:39.768189 18578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:39.768250 18581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:07:39.768618 18573 server_base.cc:1034] running on GCE node
I20260925 22:07:39.768862 18573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:07:39.769138 18573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:07:39.770270 18573 hybrid_clock.cc:648] HybridClock initialized: now 1790374059770260 us; error 23 us; skew 500 ppm
I20260925 22:07:39.771978 18491 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.0.124" port: 43427 } attempt: 1
Sep 25 22:07:39 dist-test-slave-7mqr krb5kdc[18410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374059, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:07:39.772380 18493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.772256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:43427 (local address 127.0.0.1:39892)
0925 22:07:39.772310 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:39.772339 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.124:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260925 22:07:39.772782 18493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.772664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:43427 (local address 127.0.0.1:39908)
0925 22:07:39.772721 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:39.772760 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.124:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260925 22:07:39.772828 18573 init.cc:377] Logged in from keytab as kudu/127.5.0.124@KRBTEST.COM (short username kudu)
W20260925 22:07:39.772897 18491 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.0.124:43427: Network error: Client connection negotiation failed: client connection to 127.5.0.124:43427: connect: Connection refused (error 111)
I20260925 22:07:39.773293 18573 webserver.cc:460] Webserver started at http://127.5.0.124:45407/ using document root <none> and password file <none>
I20260925 22:07:39.773542 18573 fs_manager.cc:359] Metadata directory not provided
I20260925 22:07:39.773612 18573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:07:39.773797 18573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:07:39.774812 18573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-2/data/instance:
uuid: "96c21f469546461dadbdab3673b98005"
format_stamp: "Formatted at 2026-09-25 22:07:39 on dist-test-slave-7mqr"
server_key: "a356f14cb3e35e5031a734318221fea7"
server_key_iv: "b8242fe1e96740a50a63f023145464c3"
server_key_version: "encryptionkey@0"
I20260925 22:07:39.775168 18573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-2/wal/instance:
uuid: "96c21f469546461dadbdab3673b98005"
format_stamp: "Formatted at 2026-09-25 22:07:39 on dist-test-slave-7mqr"
server_key: "a356f14cb3e35e5031a734318221fea7"
server_key_iv: "b8242fe1e96740a50a63f023145464c3"
server_key_version: "encryptionkey@0"
I20260925 22:07:39.776803 18573 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:07:39.777666 18588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:39.777841 18573 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:07:39.777931 18573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-2/wal
uuid: "96c21f469546461dadbdab3673b98005"
format_stamp: "Formatted at 2026-09-25 22:07:39 on dist-test-slave-7mqr"
server_key: "a356f14cb3e35e5031a734318221fea7"
server_key_iv: "b8242fe1e96740a50a63f023145464c3"
server_key_version: "encryptionkey@0"
I20260925 22:07:39.777990 18573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:07:39.790599 18566 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.0.124" port: 43427 } attempt: 2
I20260925 22:07:39.791044 18569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.790880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:43427 (local address 127.0.0.1:39922)
0925 22:07:39.790954 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:39.790991 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.124:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260925 22:07:39.791404 18569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.791300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:43427 (local address 127.0.0.1:39930)
0925 22:07:39.791359 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:39.791386 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.124:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260925 22:07:39.791488 18566 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.0.124:43427: Network error: Client connection negotiation failed: client connection to 127.5.0.124:43427: connect: Connection refused (error 111)
I20260925 22:07:39.796021 18573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:07:39.797823 18573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:07:39.797982 18573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:07:39.801831 18573 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.124:43427
I20260925 22:07:39.801873 18640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.124:43427 every 8 connection(s)
I20260925 22:07:39.802250 18573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-2/data/info.pb
I20260925 22:07:39.803118 18641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:07:39.803174 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 18573
I20260925 22:07:39.803282 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373908987421-5121-0/minicluster-data/master-2/wal/instance
I20260925 22:07:39.803506 5121 external_mini_cluster.cc:1428] Setting key 897cdb6699c9747a1b8d1e1ba80bd48d
I20260925 22:07:39.805472 18641 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43827 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 22:07:39 dist-test-slave-7mqr krb5kdc[18410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.124@KRBTEST.COM
Sep 25 22:07:39 dist-test-slave-7mqr krb5kdc[18410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374059, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.124@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:07:39.814002 18494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.806181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54418 (local address 127.5.0.126:43827)
0925 22:07:39.806255 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:39.806258 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:39.806403 (+ 145us) server_negotiation.cc:407] Connection header received
0925 22:07:39.806485 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:39.806486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:39.806495 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:39.806545 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:39.807063 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.807829 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.808932 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.809172 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.809259 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:39.811453 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:39.811454 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:39.811454 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:39.811462 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:39.812771 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:39.813147 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:39.813148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:39.813148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:39.813205 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:39.813434 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:39.813435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:39.813435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:39.813548 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:39.813624 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:39.813827 (+ 203us) server_negotiation.cc:299] Negotiation successful
0925 22:07:39.813904 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 22:07:39.814354 18645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.806125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:43827 (local address 127.0.0.1:54418)
0925 22:07:39.806299 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:39.806337 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:39.806406 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:39.806577 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:39.806580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:39.806902 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:07:39.806982 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:39.806984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.807961 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.807961 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:39.808847 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:39.808849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.809265 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.809266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:39.809337 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:39.809786 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:07:39.809794 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:07:39.811357 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:07:39.812862 (+ 1505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:39.812868 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:39.812870 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:39.813065 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:39.813274 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:39.813275 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:39.813275 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:39.813365 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:39.813616 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:07:39.813621 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:07:39.813708 (+ 87us) client_negotiation.cc:769] Sending connection context
0925 22:07:39.813806 (+ 98us) client_negotiation.cc:240] Negotiation successful
0925 22:07:39.813964 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":105,"threads_started":1}
I20260925 22:07:39.814355 18644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.804033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39946 (local address 127.5.0.124:43427)
0925 22:07:39.804221 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:39.804230 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:39.804261 (+ 31us) server_negotiation.cc:407] Connection header received
0925 22:07:39.804760 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:39.804769 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:39.804834 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:39.804984 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:39.805595 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.806928 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.808017 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.808605 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.808721 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:39.811034 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:39.811043 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:39.811043 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:39.811050 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:39.812790 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:39.813179 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:39.813182 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:39.813182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:39.813263 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:39.813477 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:39.813478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:39.813478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:39.813682 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:39.813847 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:39.813914 (+ 67us) server_negotiation.cc:299] Negotiation successful
0925 22:07:39.814036 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":144,"thread_start_us":62,"threads_started":1}
I20260925 22:07:39.814924 18641 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.0.125" port: 46391 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 22:07:39 dist-test-slave-7mqr krb5kdc[18410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374059, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.124@KRBTEST.COM for kudu/127.5.0.125@KRBTEST.COM
Sep 25 22:07:39 dist-test-slave-7mqr krb5kdc[18410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:07:39.822278 18645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.815231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:46391 (local address 127.0.0.1:41806)
0925 22:07:39.815280 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:39.815291 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:39.815328 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:39.815465 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:39.815465 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:39.815657 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:07:39.815694 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:39.815696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.816507 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.816507 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:39.817069 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:39.817071 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.817504 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.817505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:39.817559 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:39.817986 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:07:39.817990 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:07:39.819379 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:07:39.821155 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:39.821156 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:39.821157 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:39.821319 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:39.821561 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:39.821562 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:39.821562 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:39.821640 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:39.821936 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:07:39.821937 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:07:39.821985 (+ 48us) client_negotiation.cc:769] Sending connection context
0925 22:07:39.822059 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 22:07:39.822153 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260925 22:07:39.822301 18571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.815313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41806 (local address 127.5.0.125:46391)
0925 22:07:39.815360 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:39.815362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:39.815370 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:07:39.815397 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:39.815398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:39.815409 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:39.815464 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:39.815772 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.816436 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.817144 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.817438 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.817500 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:39.819466 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:39.819467 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:39.819468 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:39.819474 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:39.821058 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:39.821398 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:39.821399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:39.821399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:39.821459 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:39.821718 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:39.821719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:39.821719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:39.821850 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:39.821916 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:39.822113 (+ 197us) server_negotiation.cc:299] Negotiation successful
0925 22:07:39.822205 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 22:07:39.822700 18641 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.0.124" port: 43427 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:07:39.822782 18494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.815360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54430 (local address 127.5.0.126:43827)
0925 22:07:39.815393 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:39.815394 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:39.815401 (+ 7us) server_negotiation.cc:407] Connection header received
0925 22:07:39.815532 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:39.815532 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:39.815542 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:39.815604 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:39.816168 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.816667 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.817404 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.817654 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.817719 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:39.820151 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:39.820153 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:39.820153 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:39.820160 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:39.821558 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:39.821936 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:39.821937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:39.821937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:39.821996 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:39.822227 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:39.822228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:39.822229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:39.822356 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:39.822557 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:39.822595 (+ 38us) server_negotiation.cc:299] Negotiation successful
0925 22:07:39.822668 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Sep 25 22:07:39 dist-test-slave-7mqr krb5kdc[18410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374059, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.124@KRBTEST.COM for kudu/127.5.0.124@KRBTEST.COM
Sep 25 22:07:39 dist-test-slave-7mqr krb5kdc[18410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.125@KRBTEST.COM
I20260925 22:07:39.829937 18644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.823015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39962 (local address 127.5.0.124:43427)
0925 22:07:39.823049 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:39.823051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:39.823058 (+ 7us) server_negotiation.cc:407] Connection header received
0925 22:07:39.823075 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:39.823075 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:39.823084 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:39.823172 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:39.823468 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.824253 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.825123 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.825430 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.825498 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:39.827402 (+ 1904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:39.827403 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:39.827404 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:39.827410 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:39.828818 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:39.829198 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:39.829199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:39.829199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:39.829247 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:39.829449 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:39.829450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:39.829450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:39.829587 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:39.829656 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:39.829814 (+ 158us) server_negotiation.cc:299] Negotiation successful
0925 22:07:39.829877 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 22:07:39.830052 18645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.822938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:43427 (local address 127.0.0.1:39962)
0925 22:07:39.822978 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:39.822989 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:39.823023 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:39.823206 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:39.823207 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:39.823362 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:07:39.823391 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:39.823393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.824333 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.824334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:39.825056 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:39.825058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.825500 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.825501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:39.825538 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:39.825925 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:07:39.825930 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:07:39.827318 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:07:39.828899 (+ 1581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:39.828899 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:39.828900 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:39.829121 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:39.829318 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:39.829319 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:39.829319 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:39.829382 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:39.829661 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:07:39.829662 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:07:39.829714 (+ 52us) client_negotiation.cc:769] Sending connection context
0925 22:07:39.829799 (+ 85us) client_negotiation.cc:240] Negotiation successful
0925 22:07:39.829951 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":62}
I20260925 22:07:39.831357 18571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.823435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41810 (local address 127.5.0.125:46391)
0925 22:07:39.823474 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:39.823476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:39.823535 (+ 59us) server_negotiation.cc:407] Connection header received
0925 22:07:39.823583 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:39.823584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:39.823591 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:39.823634 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:39.824186 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.824886 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.825735 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.825933 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.826003 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:39.828596 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:39.828598 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:39.828598 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:39.828605 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:39.830313 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:39.830698 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:39.830699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:39.830699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:39.830749 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:39.830958 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:39.830958 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:39.830958 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:39.831051 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:39.831195 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:39.831243 (+ 48us) server_negotiation.cc:299] Negotiation successful
0925 22:07:39.831300 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:07:39.831835 18641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96c21f469546461dadbdab3673b98005: Bootstrap starting.
I20260925 22:07:39.832762 5121 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:07:39.832929 18641 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96c21f469546461dadbdab3673b98005: Neither blocks nor log segments found. Creating new log.
I20260925 22:07:39.833220 18641 log.cc:824] T 00000000000000000000000000000000 P 96c21f469546461dadbdab3673b98005: Log is configured to *not* fsync() on all Append() calls
I20260925 22:07:39.834151 18641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96c21f469546461dadbdab3673b98005: No bootstrap required, opened a new log
I20260925 22:07:39.836093 18641 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96c21f469546461dadbdab3673b98005 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1bb831817514aa3b3ce02af42da76a2" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43827 } } peers { permanent_uuid: "65ab35a533464e908a727861c0eb005a" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 46391 } } peers { permanent_uuid: "96c21f469546461dadbdab3673b98005" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 43427 } }
I20260925 22:07:39.836254 18641 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96c21f469546461dadbdab3673b98005 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:07:39.836299 18641 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96c21f469546461dadbdab3673b98005 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96c21f469546461dadbdab3673b98005, State: Initialized, Role: FOLLOWER
I20260925 22:07:39.836427 18641 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96c21f469546461dadbdab3673b98005 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1bb831817514aa3b3ce02af42da76a2" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43827 } } peers { permanent_uuid: "65ab35a533464e908a727861c0eb005a" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 46391 } } peers { permanent_uuid: "96c21f469546461dadbdab3673b98005" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 43427 } }
I20260925 22:07:39.836721 18641 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96c21f469546461dadbdab3673b98005 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:07:39.839208 18654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96c21f469546461dadbdab3673b98005 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1bb831817514aa3b3ce02af42da76a2" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43827 } } peers { permanent_uuid: "65ab35a533464e908a727861c0eb005a" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 46391 } } peers { permanent_uuid: "96c21f469546461dadbdab3673b98005" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 43427 } } }
I20260925 22:07:39.839329 18654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96c21f469546461dadbdab3673b98005 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 22:07:39.840072 18665 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 96c21f469546461dadbdab3673b98005: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:07:39.840145 18665 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:07:39.842054 18494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.834043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54444 (local address 127.5.0.126:43827)
0925 22:07:39.834086 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:39.834088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:39.834215 (+ 127us) server_negotiation.cc:407] Connection header received
0925 22:07:39.834296 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:39.834297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:39.834308 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:39.834362 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:39.835082 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.835781 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.837086 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.837400 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.837446 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:39.838915 (+ 1469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:39.838917 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:39.838917 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:39.838925 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:39.840909 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:39.841253 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:39.841255 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:39.841256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:39.841315 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:39.841521 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:39.841522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:39.841522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:39.841658 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:39.841763 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:39.841869 (+ 106us) server_negotiation.cc:299] Negotiation successful
0925 22:07:39.841946 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:07:39.843732 18644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.834268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39964 (local address 127.5.0.124:43427)
0925 22:07:39.834334 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:39.834336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:39.835211 (+ 875us) server_negotiation.cc:407] Connection header received
0925 22:07:39.835300 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:39.835301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:39.835310 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:39.835337 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:39.835995 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.836704 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.838232 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.838551 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.838592 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:39.840598 (+ 2006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:39.840599 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:39.840599 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:39.840606 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:39.842598 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:39.842927 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:39.842928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:39.842928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:39.842981 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:39.843203 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:39.843204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:39.843204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:39.843336 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:39.843435 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:39.843546 (+ 111us) server_negotiation.cc:299] Negotiation successful
0925 22:07:39.843618 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 22:07:39.840298 18573 master_runner.cc:186] Error getting master registration for 127.5.0.126:43827: OK, instance_id { permanent_uuid: "d1bb831817514aa3b3ce02af42da76a2" instance_seqno: 1790374059505828 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 22:07:39.844187 18573 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 22:07:39.845878 18571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.834187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41816 (local address 127.5.0.125:46391)
0925 22:07:39.834219 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:39.834222 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:39.834376 (+ 154us) server_negotiation.cc:407] Connection header received
0925 22:07:39.834466 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:39.834467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:39.834476 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:39.834506 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:39.835122 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.835862 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.836911 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.837187 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.837244 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:39.838925 (+ 1681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:39.838927 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:39.838927 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:39.838933 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:39.844448 (+ 5515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:39.844761 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:39.844763 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:39.844763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:39.844828 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:39.844995 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:39.844996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:39.844996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:39.845127 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:39.845653 (+ 526us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:39.845696 (+ 43us) server_negotiation.cc:299] Negotiation successful
0925 22:07:39.845781 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 22:07:39.848009 18491 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.0.124" port: 43427 } attempt: 2
Sep 25 22:07:39 dist-test-slave-7mqr krb5kdc[18410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374059, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for kudu/127.5.0.124@KRBTEST.COM
I20260925 22:07:39.855031 18493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.848294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:43427 (local address 127.0.0.1:39976)
0925 22:07:39.848354 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:39.848366 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:39.848412 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:39.848652 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:39.848654 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:39.848811 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:07:39.848856 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:39.848857 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.849437 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.849438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:39.850277 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:39.850279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.850673 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.850674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:39.850730 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:39.851117 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:07:39.851122 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:07:39.852563 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:07:39.854026 (+ 1463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:39.854027 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:39.854028 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:39.854204 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:39.854411 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:39.854412 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:39.854412 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:39.854467 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:39.854717 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:07:39.854719 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:07:39.854760 (+ 41us) client_negotiation.cc:769] Sending connection context
0925 22:07:39.854847 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 22:07:39.854936 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260925 22:07:39.855058 18644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.848501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39976 (local address 127.5.0.124:43427)
0925 22:07:39.848551 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:39.848553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:39.848559 (+ 6us) server_negotiation.cc:407] Connection header received
0925 22:07:39.848575 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:39.848576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:39.848583 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:39.848631 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:39.848926 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.849360 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.850362 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.850601 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.850653 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:39.852650 (+ 1997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:39.852652 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:39.852652 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:39.852659 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:39.853942 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:39.854288 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:39.854289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:39.854289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:39.854342 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:39.854539 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:39.854540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:39.854541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:39.854649 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:39.854724 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:39.854871 (+ 147us) server_negotiation.cc:299] Negotiation successful
0925 22:07:39.854936 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 22:07:39.856931 18491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1bb831817514aa3b3ce02af42da76a2: Bootstrap starting.
I20260925 22:07:39.857673 18491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1bb831817514aa3b3ce02af42da76a2: Neither blocks nor log segments found. Creating new log.
I20260925 22:07:39.857975 18491 log.cc:824] T 00000000000000000000000000000000 P d1bb831817514aa3b3ce02af42da76a2: Log is configured to *not* fsync() on all Append() calls
I20260925 22:07:39.858767 18491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1bb831817514aa3b3ce02af42da76a2: No bootstrap required, opened a new log
I20260925 22:07:39.860755 18491 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1bb831817514aa3b3ce02af42da76a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1bb831817514aa3b3ce02af42da76a2" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43827 } } peers { permanent_uuid: "65ab35a533464e908a727861c0eb005a" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 46391 } } peers { permanent_uuid: "96c21f469546461dadbdab3673b98005" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 43427 } }
I20260925 22:07:39.860901 18491 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1bb831817514aa3b3ce02af42da76a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:07:39.860939 18491 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1bb831817514aa3b3ce02af42da76a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1bb831817514aa3b3ce02af42da76a2, State: Initialized, Role: FOLLOWER
I20260925 22:07:39.861083 18491 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1bb831817514aa3b3ce02af42da76a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1bb831817514aa3b3ce02af42da76a2" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43827 } } peers { permanent_uuid: "65ab35a533464e908a727861c0eb005a" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 46391 } } peers { permanent_uuid: "96c21f469546461dadbdab3673b98005" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 43427 } }
I20260925 22:07:39.861347 18668 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1bb831817514aa3b3ce02af42da76a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1bb831817514aa3b3ce02af42da76a2" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43827 } } peers { permanent_uuid: "65ab35a533464e908a727861c0eb005a" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 46391 } } peers { permanent_uuid: "96c21f469546461dadbdab3673b98005" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 43427 } } }
I20260925 22:07:39.861399 18491 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1bb831817514aa3b3ce02af42da76a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:07:39.861449 18668 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1bb831817514aa3b3ce02af42da76a2 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 22:07:39.863184 18679 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d1bb831817514aa3b3ce02af42da76a2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:07:39.863283 18679 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:07:39.863786 18423 master_runner.cc:186] Error getting master registration for 127.5.0.125:46391: OK, instance_id { permanent_uuid: "65ab35a533464e908a727861c0eb005a" instance_seqno: 1790374059648418 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 22:07:39.865793 18423 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 22:07:39.882630 18566 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.0.124" port: 43427 } attempt: 3
Sep 25 22:07:39 dist-test-slave-7mqr krb5kdc[18410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374059, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.125@KRBTEST.COM for kudu/127.5.0.124@KRBTEST.COM
I20260925 22:07:39.890354 18569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.883052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:43427 (local address 127.0.0.1:39986)
0925 22:07:39.883103 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:39.883115 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:39.883162 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:39.883309 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:39.883310 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:39.883493 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:07:39.883536 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:39.883538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.884383 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.884385 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:39.885029 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:39.885032 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.885399 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.885400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:39.885500 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:39.885997 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:07:39.886003 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:07:39.887447 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:07:39.889293 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:39.889294 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:39.889294 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:39.889483 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:39.889698 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:39.889698 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:39.889699 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:39.889759 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:39.890023 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:07:39.890024 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:07:39.890070 (+ 46us) client_negotiation.cc:769] Sending connection context
0925 22:07:39.890160 (+ 90us) client_negotiation.cc:240] Negotiation successful
0925 22:07:39.890259 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260925 22:07:39.890354 18644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:39.883115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39986 (local address 127.5.0.124:43427)
0925 22:07:39.883155 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:39.883156 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:39.883165 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:07:39.883216 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:39.883217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:39.883228 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:39.883305 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:39.883739 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.884297 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.885130 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:39.885326 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:39.885403 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:39.887537 (+ 2134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:39.887539 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:39.887539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:39.887557 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:39.889211 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:39.889571 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:39.889572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:39.889573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:39.889630 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:39.889831 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:39.889832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:39.889832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:39.889946 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:39.890031 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:39.890185 (+ 154us) server_negotiation.cc:299] Negotiation successful
0925 22:07:39.890259 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:07:39.892046 18566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65ab35a533464e908a727861c0eb005a: Bootstrap starting.
I20260925 22:07:39.892704 18566 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65ab35a533464e908a727861c0eb005a: Neither blocks nor log segments found. Creating new log.
I20260925 22:07:39.893005 18566 log.cc:824] T 00000000000000000000000000000000 P 65ab35a533464e908a727861c0eb005a: Log is configured to *not* fsync() on all Append() calls
I20260925 22:07:39.893780 18566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65ab35a533464e908a727861c0eb005a: No bootstrap required, opened a new log
I20260925 22:07:39.895174 18566 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65ab35a533464e908a727861c0eb005a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1bb831817514aa3b3ce02af42da76a2" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43827 } } peers { permanent_uuid: "65ab35a533464e908a727861c0eb005a" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 46391 } } peers { permanent_uuid: "96c21f469546461dadbdab3673b98005" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 43427 } }
I20260925 22:07:39.895308 18566 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65ab35a533464e908a727861c0eb005a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:07:39.895352 18566 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65ab35a533464e908a727861c0eb005a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65ab35a533464e908a727861c0eb005a, State: Initialized, Role: FOLLOWER
I20260925 22:07:39.895495 18566 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65ab35a533464e908a727861c0eb005a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1bb831817514aa3b3ce02af42da76a2" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43827 } } peers { permanent_uuid: "65ab35a533464e908a727861c0eb005a" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 46391 } } peers { permanent_uuid: "96c21f469546461dadbdab3673b98005" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 43427 } }
I20260925 22:07:39.895768 18682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65ab35a533464e908a727861c0eb005a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1bb831817514aa3b3ce02af42da76a2" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43827 } } peers { permanent_uuid: "65ab35a533464e908a727861c0eb005a" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 46391 } } peers { permanent_uuid: "96c21f469546461dadbdab3673b98005" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 43427 } } }
I20260925 22:07:39.895922 18682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65ab35a533464e908a727861c0eb005a [sys.catalog]: This master's current role is: FOLLOWER
W20260925 22:07:39.896224 18519 tablet.cc:2369] T 00000000000000000000000000000000 P 65ab35a533464e908a727861c0eb005a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:07:39.895833 18566 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65ab35a533464e908a727861c0eb005a [sys.catalog]: configured and running, proceeding with master startup.
W20260925 22:07:39.898393 18693 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 65ab35a533464e908a727861c0eb005a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:07:39.898520 18693 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:07:39.900552 18498 master_runner.cc:263] No leader master found from master 65ab35a533464e908a727861c0eb005a
I20260925 22:07:39.900600 18498 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 22:07:39.969805 18654 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96c21f469546461dadbdab3673b98005 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:07:39.969939 18654 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96c21f469546461dadbdab3673b98005 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1bb831817514aa3b3ce02af42da76a2" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43827 } } peers { permanent_uuid: "65ab35a533464e908a727861c0eb005a" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 46391 } } peers { permanent_uuid: "96c21f469546461dadbdab3673b98005" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 43427 } }
I20260925 22:07:39.970196 18654 leader_election.cc:288] T 00000000000000000000000000000000 P 96c21f469546461dadbdab3673b98005 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1bb831817514aa3b3ce02af42da76a2 (127.5.0.126:43827), 65ab35a533464e908a727861c0eb005a (127.5.0.125:46391)
I20260925 22:07:39.970467 18465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "96c21f469546461dadbdab3673b98005" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1bb831817514aa3b3ce02af42da76a2" is_pre_election: true
I20260925 22:07:39.970618 18465 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d1bb831817514aa3b3ce02af42da76a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96c21f469546461dadbdab3673b98005 in term 0.
I20260925 22:07:39.970994 18540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "96c21f469546461dadbdab3673b98005" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65ab35a533464e908a727861c0eb005a" is_pre_election: true
I20260925 22:07:39.971131 18540 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 65ab35a533464e908a727861c0eb005a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96c21f469546461dadbdab3673b98005 in term 0.
I20260925 22:07:39.971395 18592 leader_election.cc:302] T 00000000000000000000000000000000 P 96c21f469546461dadbdab3673b98005 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96c21f469546461dadbdab3673b98005, d1bb831817514aa3b3ce02af42da76a2; no voters:
I20260925 22:07:39.971580 18654 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96c21f469546461dadbdab3673b98005 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:07:39.971634 18654 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96c21f469546461dadbdab3673b98005 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:07:39.971658 18654 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96c21f469546461dadbdab3673b98005 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:07:39.972649 18654 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96c21f469546461dadbdab3673b98005 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1bb831817514aa3b3ce02af42da76a2" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43827 } } peers { permanent_uuid: "65ab35a533464e908a727861c0eb005a" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 46391 } } peers { permanent_uuid: "96c21f469546461dadbdab3673b98005" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 43427 } }
I20260925 22:07:39.972774 18654 leader_election.cc:288] T 00000000000000000000000000000000 P 96c21f469546461dadbdab3673b98005 [CANDIDATE]: Term 1 election: Requested vote from peers d1bb831817514aa3b3ce02af42da76a2 (127.5.0.126:43827), 65ab35a533464e908a727861c0eb005a (127.5.0.125:46391)
I20260925 22:07:39.973219 18540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "96c21f469546461dadbdab3673b98005" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65ab35a533464e908a727861c0eb005a"
I20260925 22:07:39.973297 18540 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65ab35a533464e908a727861c0eb005a [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:07:39.974169 18540 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 65ab35a533464e908a727861c0eb005a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96c21f469546461dadbdab3673b98005 in term 1.
I20260925 22:07:39.974390 18465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "96c21f469546461dadbdab3673b98005" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1bb831817514aa3b3ce02af42da76a2"
I20260925 22:07:39.974467 18465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1bb831817514aa3b3ce02af42da76a2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:07:39.975273 18465 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d1bb831817514aa3b3ce02af42da76a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96c21f469546461dadbdab3673b98005 in term 1.
I20260925 22:07:39.975488 18592 leader_election.cc:302] T 00000000000000000000000000000000 P 96c21f469546461dadbdab3673b98005 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96c21f469546461dadbdab3673b98005, d1bb831817514aa3b3ce02af42da76a2; no voters:
I20260925 22:07:39.975708 18654 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96c21f469546461dadbdab3673b98005 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:07:39.975863 18654 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96c21f469546461dadbdab3673b98005 [term 1 LEADER]: Becoming Leader. State: Replica: 96c21f469546461dadbdab3673b98005, State: Running, Role: LEADER
I20260925 22:07:39.975977 18654 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96c21f469546461dadbdab3673b98005 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1bb831817514aa3b3ce02af42da76a2" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43827 } } peers { permanent_uuid: "65ab35a533464e908a727861c0eb005a" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 46391 } } peers { permanent_uuid: "96c21f469546461dadbdab3673b98005" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 43427 } }
I20260925 22:07:39.976342 18654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96c21f469546461dadbdab3673b98005 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96c21f469546461dadbdab3673b98005. Latest consensus state: current_term: 1 leader_uuid: "96c21f469546461dadbdab3673b98005" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1bb831817514aa3b3ce02af42da76a2" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43827 } } peers { permanent_uuid: "65ab35a533464e908a727861c0eb005a" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 46391 } } peers { permanent_uuid: "96c21f469546461dadbdab3673b98005" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 43427 } } }
I20260925 22:07:39.976424 18654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96c21f469546461dadbdab3673b98005 [sys.catalog]: This master's current role is: LEADER
I20260925 22:07:39.977316 18697 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:07:39.977553 18697 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:07:39.979036 18540 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 65ab35a533464e908a727861c0eb005a [term 1 FOLLOWER]: Refusing update from remote peer 96c21f469546461dadbdab3673b98005: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:07:39.979399 18695 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 96c21f469546461dadbdab3673b98005 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65ab35a533464e908a727861c0eb005a" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 46391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:07:39.981134 18682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65ab35a533464e908a727861c0eb005a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96c21f469546461dadbdab3673b98005. Latest consensus state: current_term: 1 leader_uuid: "96c21f469546461dadbdab3673b98005" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1bb831817514aa3b3ce02af42da76a2" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43827 } } peers { permanent_uuid: "65ab35a533464e908a727861c0eb005a" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 46391 } } peers { permanent_uuid: "96c21f469546461dadbdab3673b98005" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 43427 } } }
I20260925 22:07:39.981220 18682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65ab35a533464e908a727861c0eb005a [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:07:39.981739 18465 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d1bb831817514aa3b3ce02af42da76a2 [term 1 FOLLOWER]: Refusing update from remote peer 96c21f469546461dadbdab3673b98005: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:07:39.982038 18695 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 96c21f469546461dadbdab3673b98005 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1bb831817514aa3b3ce02af42da76a2" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:07:39.983213 18668 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1bb831817514aa3b3ce02af42da76a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96c21f469546461dadbdab3673b98005. Latest consensus state: current_term: 1 leader_uuid: "96c21f469546461dadbdab3673b98005" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1bb831817514aa3b3ce02af42da76a2" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43827 } } peers { permanent_uuid: "65ab35a533464e908a727861c0eb005a" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 46391 } } peers { permanent_uuid: "96c21f469546461dadbdab3673b98005" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 43427 } } }
I20260925 22:07:39.983314 18668 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1bb831817514aa3b3ce02af42da76a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:07:39.984474 18695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96c21f469546461dadbdab3673b98005 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "96c21f469546461dadbdab3673b98005" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1bb831817514aa3b3ce02af42da76a2" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43827 } } peers { permanent_uuid: "65ab35a533464e908a727861c0eb005a" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 46391 } } peers { permanent_uuid: "96c21f469546461dadbdab3673b98005" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 43427 } } }
I20260925 22:07:39.984552 18695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96c21f469546461dadbdab3673b98005 [sys.catalog]: This master's current role is: LEADER
I20260925 22:07:39.985910 18682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65ab35a533464e908a727861c0eb005a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "96c21f469546461dadbdab3673b98005" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1bb831817514aa3b3ce02af42da76a2" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43827 } } peers { permanent_uuid: "65ab35a533464e908a727861c0eb005a" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 46391 } } peers { permanent_uuid: "96c21f469546461dadbdab3673b98005" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 43427 } } }
I20260925 22:07:39.985998 18682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65ab35a533464e908a727861c0eb005a [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:07:39.986454 18698 mvcc.cc:205] Tried to move back new op lower bound from 7333372149671927808 to 7333372149662162944. Current Snapshot: MvccSnapshot[applied={T|T < 7333372149671927808}]
I20260925 22:07:39.987257 18654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96c21f469546461dadbdab3673b98005 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "96c21f469546461dadbdab3673b98005" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1bb831817514aa3b3ce02af42da76a2" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43827 } } peers { permanent_uuid: "65ab35a533464e908a727861c0eb005a" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 46391 } } peers { permanent_uuid: "96c21f469546461dadbdab3673b98005" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 43427 } } }
I20260925 22:07:39.987332 18654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96c21f469546461dadbdab3673b98005 [sys.catalog]: This master's current role is: LEADER
I20260925 22:07:39.987685 18668 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1bb831817514aa3b3ce02af42da76a2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "96c21f469546461dadbdab3673b98005" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1bb831817514aa3b3ce02af42da76a2" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43827 } } peers { permanent_uuid: "65ab35a533464e908a727861c0eb005a" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 46391 } } peers { permanent_uuid: "96c21f469546461dadbdab3673b98005" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 43427 } } }
I20260925 22:07:39.987757 18668 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1bb831817514aa3b3ce02af42da76a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:07:39.988296 18697 catalog_manager.cc:1347] Generated new cluster ID: 959ed47ecc9341a69b7dd22c9897ea70
I20260925 22:07:39.988341 18697 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:07:39.995945 18699 mvcc.cc:205] Tried to move back new op lower bound from 7333372149671927808 to 7333372149662162944. Current Snapshot: MvccSnapshot[applied={T|T < 7333372149671927808}]
I20260925 22:07:39.998077 18701 mvcc.cc:205] Tried to move back new op lower bound from 7333372149671927808 to 7333372149662162944. Current Snapshot: MvccSnapshot[applied={T|T < 7333372149671927808}]
I20260925 22:07:40.001261 18697 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:07:40.002091 18697 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:07:40.013842 18697 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96c21f469546461dadbdab3673b98005: Generated new TSK 0
I20260925 22:07:40.014142 18697 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:07:40.863755 18679 catalog_manager.cc:1259] Loaded cluster ID: 959ed47ecc9341a69b7dd22c9897ea70
I20260925 22:07:40.863833 18679 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d1bb831817514aa3b3ce02af42da76a2: loading cluster ID for follower catalog manager: success
I20260925 22:07:40.864687 18679 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d1bb831817514aa3b3ce02af42da76a2: acquiring CA information for follower catalog manager: success
I20260925 22:07:40.865001 18679 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d1bb831817514aa3b3ce02af42da76a2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 22:07:40.898972 18693 catalog_manager.cc:1259] Loaded cluster ID: 959ed47ecc9341a69b7dd22c9897ea70
I20260925 22:07:40.899049 18693 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 65ab35a533464e908a727861c0eb005a: loading cluster ID for follower catalog manager: success
I20260925 22:07:40.899868 18693 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 65ab35a533464e908a727861c0eb005a: acquiring CA information for follower catalog manager: success
I20260925 22:07:40.900211 18693 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 65ab35a533464e908a727861c0eb005a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 22:07:41.078339 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 18423
I20260925 22:07:41.083639 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 18498
I20260925 22:07:41.088702 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 18573
2026-09-25T22:07:41Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2849 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:07:41 dist-test-slave-7mqr krb5kdc[18714](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:07:41 dist-test-slave-7mqr krb5kdc[18714](info): set up 2 sockets
Sep 25 22:07:41 dist-test-slave-7mqr krb5kdc[18714](info): commencing operation
krb5kdc: starting...
W20260925 22:07:42.132367 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:07:42 dist-test-slave-7mqr krb5kdc[18714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:07:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:07:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.0.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
I20260925 22:07:42.251502 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:46155
--webserver_interface=127.5.0.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:44767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:46155
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:07:42.331425 18727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:07:42.331674 18727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:07:42.331715 18727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:07:42.332906 18727 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:07:42.332978 18727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:07:42.333000 18727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:07:42.333022 18727 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:07:42.333053 18727 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:07:42.334488 18727 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:44767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:46155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:46155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:07:42.334769 18727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:07:42.334990 18727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:07:42.339581 18732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:42.339615 18733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:42.339672 18735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:07:42.339860 18727 server_base.cc:1034] running on GCE node
I20260925 22:07:42.340134 18727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:07:42.340415 18727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:07:42.341598 18727 hybrid_clock.cc:648] HybridClock initialized: now 1790374062341593 us; error 39 us; skew 500 ppm
Sep 25 22:07:42 dist-test-slave-7mqr krb5kdc[18714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374062, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:07:42.344421 18727 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:07:42.344944 18727 webserver.cc:460] Webserver started at http://127.5.0.126:32837/ using document root <none> and password file <none>
I20260925 22:07:42.345225 18727 fs_manager.cc:359] Metadata directory not provided
I20260925 22:07:42.345281 18727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:07:42.345410 18727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:07:42.347016 18727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "872d0c8eb38d49e6b924853711b66e9f"
format_stamp: "Formatted at 2026-09-25 22:07:42 on dist-test-slave-7mqr"
server_key: "02479a4d6189c5aff8a9eff7713d0549"
server_key_iv: "664f3d6e83525ab0031a94e3f02a40bc"
server_key_version: "encryptionkey@0"
I20260925 22:07:42.347447 18727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "872d0c8eb38d49e6b924853711b66e9f"
format_stamp: "Formatted at 2026-09-25 22:07:42 on dist-test-slave-7mqr"
server_key: "02479a4d6189c5aff8a9eff7713d0549"
server_key_iv: "664f3d6e83525ab0031a94e3f02a40bc"
server_key_version: "encryptionkey@0"
I20260925 22:07:42.349720 18727 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260925 22:07:42.351446 18742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:42.351665 18727 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:07:42.351769 18727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "872d0c8eb38d49e6b924853711b66e9f"
format_stamp: "Formatted at 2026-09-25 22:07:42 on dist-test-slave-7mqr"
server_key: "02479a4d6189c5aff8a9eff7713d0549"
server_key_iv: "664f3d6e83525ab0031a94e3f02a40bc"
server_key_version: "encryptionkey@0"
I20260925 22:07:42.351850 18727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:07:42.375757 18727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:07:42.380097 18727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:07:42.380267 18727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:07:42.386384 18727 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:46155
I20260925 22:07:42.386461 18794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:46155 every 8 connection(s)
I20260925 22:07:42.386835 18727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:07:42.387743 18795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:07:42.390182 18795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 872d0c8eb38d49e6b924853711b66e9f: Bootstrap starting.
I20260925 22:07:42.390766 18795 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 872d0c8eb38d49e6b924853711b66e9f: Neither blocks nor log segments found. Creating new log.
I20260925 22:07:42.391054 18795 log.cc:824] T 00000000000000000000000000000000 P 872d0c8eb38d49e6b924853711b66e9f: Log is configured to *not* fsync() on all Append() calls
I20260925 22:07:42.391747 18795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 872d0c8eb38d49e6b924853711b66e9f: No bootstrap required, opened a new log
I20260925 22:07:42.393594 18795 raft_consensus.cc:348] T 00000000000000000000000000000000 P 872d0c8eb38d49e6b924853711b66e9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "872d0c8eb38d49e6b924853711b66e9f" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 46155 } }
I20260925 22:07:42.393729 18795 raft_consensus.cc:374] T 00000000000000000000000000000000 P 872d0c8eb38d49e6b924853711b66e9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:07:42.393754 18795 raft_consensus.cc:729] T 00000000000000000000000000000000 P 872d0c8eb38d49e6b924853711b66e9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 872d0c8eb38d49e6b924853711b66e9f, State: Initialized, Role: FOLLOWER
I20260925 22:07:42.393924 18795 consensus_queue.cc:260] T 00000000000000000000000000000000 P 872d0c8eb38d49e6b924853711b66e9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "872d0c8eb38d49e6b924853711b66e9f" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 46155 } }
I20260925 22:07:42.393985 18795 raft_consensus.cc:388] T 00000000000000000000000000000000 P 872d0c8eb38d49e6b924853711b66e9f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:07:42.394024 18795 raft_consensus.cc:482] T 00000000000000000000000000000000 P 872d0c8eb38d49e6b924853711b66e9f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:07:42.394074 18795 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 872d0c8eb38d49e6b924853711b66e9f [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:07:42.394593 18795 raft_consensus.cc:504] T 00000000000000000000000000000000 P 872d0c8eb38d49e6b924853711b66e9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "872d0c8eb38d49e6b924853711b66e9f" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 46155 } }
I20260925 22:07:42.394706 18795 leader_election.cc:302] T 00000000000000000000000000000000 P 872d0c8eb38d49e6b924853711b66e9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 872d0c8eb38d49e6b924853711b66e9f; no voters:
I20260925 22:07:42.394887 18795 leader_election.cc:288] T 00000000000000000000000000000000 P 872d0c8eb38d49e6b924853711b66e9f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:07:42.394924 18798 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 872d0c8eb38d49e6b924853711b66e9f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:07:42.395107 18798 raft_consensus.cc:686] T 00000000000000000000000000000000 P 872d0c8eb38d49e6b924853711b66e9f [term 1 LEADER]: Becoming Leader. State: Replica: 872d0c8eb38d49e6b924853711b66e9f, State: Running, Role: LEADER
I20260925 22:07:42.395138 18795 sys_catalog.cc:564] T 00000000000000000000000000000000 P 872d0c8eb38d49e6b924853711b66e9f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:07:42.395205 18798 consensus_queue.cc:237] T 00000000000000000000000000000000 P 872d0c8eb38d49e6b924853711b66e9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "872d0c8eb38d49e6b924853711b66e9f" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 46155 } }
I20260925 22:07:42.395601 18800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 872d0c8eb38d49e6b924853711b66e9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 872d0c8eb38d49e6b924853711b66e9f. Latest consensus state: current_term: 1 leader_uuid: "872d0c8eb38d49e6b924853711b66e9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "872d0c8eb38d49e6b924853711b66e9f" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 46155 } } }
I20260925 22:07:42.395704 18800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 872d0c8eb38d49e6b924853711b66e9f [sys.catalog]: This master's current role is: LEADER
I20260925 22:07:42.395733 18799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 872d0c8eb38d49e6b924853711b66e9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "872d0c8eb38d49e6b924853711b66e9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "872d0c8eb38d49e6b924853711b66e9f" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 46155 } } }
I20260925 22:07:42.395789 18799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 872d0c8eb38d49e6b924853711b66e9f [sys.catalog]: This master's current role is: LEADER
I20260925 22:07:42.396240 18807 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:07:42.396729 18807 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:07:42.396799 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 18727
I20260925 22:07:42.396905 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:07:42.397122 5121 external_mini_cluster.cc:1428] Setting key 286db0674ba3ef85d283c5dd5b172f63
I20260925 22:07:42.399115 18807 catalog_manager.cc:1347] Generated new cluster ID: 0a50cc19155642ad842aa822c0c426ed
I20260925 22:07:42.399183 18807 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:07:42 dist-test-slave-7mqr krb5kdc[18714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:07:42.407071 18807 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:07:42.407934 18807 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:07:42.409562 18817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:42.397513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52850 (local address 127.5.0.126:46155)
0925 22:07:42.397675 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:42.397684 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:42.397904 (+ 220us) server_negotiation.cc:407] Connection header received
0925 22:07:42.398103 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:42.398111 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:42.398174 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:42.400121 (+ 1947us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:42.400197 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:42.401088 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:42.401861 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:42.402264 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:42.402382 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:42.405052 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:42.405057 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:42.405057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:42.405069 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:42.408032 (+ 2963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:42.408374 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:42.408377 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:42.408377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:42.408473 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:42.408695 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:42.408696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:42.408697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:42.408933 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:42.409109 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:42.409188 (+ 79us) server_negotiation.cc:299] Negotiation successful
0925 22:07:42.409342 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":77,"threads_started":1}
I20260925 22:07:42.410451 5121 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:07:42.415261 18807 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 872d0c8eb38d49e6b924853711b66e9f: Generated new TSK 0
I20260925 22:07:42.415416 18807 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:07:42.464085 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 18727
2026-09-25T22:07:42Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1374 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:07:42 dist-test-slave-7mqr krb5kdc[18822](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:07:42 dist-test-slave-7mqr krb5kdc[18822](info): set up 2 sockets
Sep 25 22:07:42 dist-test-slave-7mqr krb5kdc[18822](info): commencing operation
krb5kdc: starting...
W20260925 22:07:43.495872 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:07:43 dist-test-slave-7mqr krb5kdc[18822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:07:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:07:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.0.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
I20260925 22:07:43.614404 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:34019
--webserver_interface=127.5.0.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:34219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:34019
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:07:43.696647 18835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:07:43.696866 18835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:07:43.696889 18835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:07:43.698087 18835 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:07:43.698136 18835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:07:43.698150 18835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:07:43.698201 18835 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:07:43.698230 18835 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:07:43.699837 18835 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:34219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:34019
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:34019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:07:43.700150 18835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:07:43.700351 18835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:07:43.705070 18840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:43.705009 18843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:07:43.705157 18835 server_base.cc:1034] running on GCE node
W20260925 22:07:43.705050 18841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:07:43.705477 18835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:07:43.705794 18835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:07:43.706954 18835 hybrid_clock.cc:648] HybridClock initialized: now 1790374063706931 us; error 41 us; skew 500 ppm
Sep 25 22:07:43 dist-test-slave-7mqr krb5kdc[18822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374063, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:07:43.709539 18835 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:07:43.709970 18835 webserver.cc:460] Webserver started at http://127.5.0.126:44265/ using document root <none> and password file <none>
I20260925 22:07:43.710194 18835 fs_manager.cc:359] Metadata directory not provided
I20260925 22:07:43.710263 18835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:07:43.710412 18835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:07:43.711472 18835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "475894d649af404baa1eb2718c7b6572"
format_stamp: "Formatted at 2026-09-25 22:07:43 on dist-test-slave-7mqr"
server_key: "630a3b972c59128422a37a0c489e9282"
server_key_iv: "36ca75de1ef56b8114a0b21502ee1d11"
server_key_version: "encryptionkey@0"
I20260925 22:07:43.711838 18835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "475894d649af404baa1eb2718c7b6572"
format_stamp: "Formatted at 2026-09-25 22:07:43 on dist-test-slave-7mqr"
server_key: "630a3b972c59128422a37a0c489e9282"
server_key_iv: "36ca75de1ef56b8114a0b21502ee1d11"
server_key_version: "encryptionkey@0"
I20260925 22:07:43.713176 18835 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:07:43.713922 18850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:43.714097 18835 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:07:43.714196 18835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "475894d649af404baa1eb2718c7b6572"
format_stamp: "Formatted at 2026-09-25 22:07:43 on dist-test-slave-7mqr"
server_key: "630a3b972c59128422a37a0c489e9282"
server_key_iv: "36ca75de1ef56b8114a0b21502ee1d11"
server_key_version: "encryptionkey@0"
I20260925 22:07:43.714279 18835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:07:43.730506 18835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:07:43.732421 18835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:07:43.732589 18835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:07:43.736358 18835 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:34019
I20260925 22:07:43.736392 18902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:34019 every 8 connection(s)
I20260925 22:07:43.736740 18835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:07:43.737751 18903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:07:43.739138 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 18835
I20260925 22:07:43.739254 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:07:43.739470 5121 external_mini_cluster.cc:1428] Setting key 492011bd067338ae0889502662b4b8a8
I20260925 22:07:43.740461 18903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572: Bootstrap starting.
I20260925 22:07:43.741148 18903 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572: Neither blocks nor log segments found. Creating new log.
I20260925 22:07:43.741494 18903 log.cc:824] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572: Log is configured to *not* fsync() on all Append() calls
I20260925 22:07:43.742230 18903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572: No bootstrap required, opened a new log
I20260925 22:07:43.744395 18903 raft_consensus.cc:348] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "475894d649af404baa1eb2718c7b6572" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34019 } }
I20260925 22:07:43.744578 18903 raft_consensus.cc:374] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:07:43.744617 18903 raft_consensus.cc:729] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 475894d649af404baa1eb2718c7b6572, State: Initialized, Role: FOLLOWER
I20260925 22:07:43.744747 18903 consensus_queue.cc:260] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "475894d649af404baa1eb2718c7b6572" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34019 } }
I20260925 22:07:43.744829 18903 raft_consensus.cc:388] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:07:43.744851 18903 raft_consensus.cc:482] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:07:43.744920 18903 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:07:43.745456 18903 raft_consensus.cc:504] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "475894d649af404baa1eb2718c7b6572" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34019 } }
I20260925 22:07:43.745571 18903 leader_election.cc:302] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 475894d649af404baa1eb2718c7b6572; no voters:
I20260925 22:07:43.745744 18903 leader_election.cc:288] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:07:43.745779 18908 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:07:43.745887 18908 raft_consensus.cc:686] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [term 1 LEADER]: Becoming Leader. State: Replica: 475894d649af404baa1eb2718c7b6572, State: Running, Role: LEADER
I20260925 22:07:43.745966 18908 consensus_queue.cc:237] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "475894d649af404baa1eb2718c7b6572" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34019 } }
I20260925 22:07:43.745998 18903 sys_catalog.cc:564] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:07:43.746284 18909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "475894d649af404baa1eb2718c7b6572" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "475894d649af404baa1eb2718c7b6572" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34019 } } }
I20260925 22:07:43.746379 18909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [sys.catalog]: This master's current role is: LEADER
I20260925 22:07:43.746605 18912 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 25 22:07:43 dist-test-slave-7mqr krb5kdc[18822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:07:43.747195 18912 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:07:43.747640 18910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 475894d649af404baa1eb2718c7b6572. Latest consensus state: current_term: 1 leader_uuid: "475894d649af404baa1eb2718c7b6572" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "475894d649af404baa1eb2718c7b6572" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34019 } } }
I20260925 22:07:43.747720 18910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [sys.catalog]: This master's current role is: LEADER
I20260925 22:07:43.749344 18912 catalog_manager.cc:1347] Generated new cluster ID: 3fc71eb117d04324bef4c9c870cd2682
I20260925 22:07:43.749401 18912 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:07:43.750864 18906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:43.740025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38472 (local address 127.5.0.126:34019)
0925 22:07:43.740205 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:43.740213 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:43.740238 (+ 25us) server_negotiation.cc:407] Connection header received
0925 22:07:43.740673 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:43.740681 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:43.740732 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:43.740845 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:43.741681 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:43.743066 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:43.743809 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:43.744491 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:43.744600 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:43.747485 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:43.747489 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:43.747489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:43.747508 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:43.749437 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:43.749786 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:43.749788 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:43.749789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:43.749874 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:43.750085 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:43.750086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:43.750086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:43.750314 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:43.750462 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:43.750572 (+ 110us) server_negotiation.cc:299] Negotiation successful
0925 22:07:43.750716 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":88,"threads_started":1}
I20260925 22:07:43.768198 18912 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:07:43.768728 18912 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:07:43.778102 18912 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572: Generated new TSK 0
I20260925 22:07:43.778319 18912 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.0.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
I20260925 22:07:43.810642 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.65:0
--local_ip_for_outbound_sockets=127.5.0.65
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:34019
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:34219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:07:43.906169 18929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:07:43.906374 18929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:07:43.906397 18929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:07:43.907555 18929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:07:43.907598 18929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:07:43.907655 18929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.65
I20260925 22:07:43.909165 18929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:34219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:34019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.65
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:07:43.909421 18929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:07:43.909662 18929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:07:43.914449 18935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:43.914506 18937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:43.914649 18934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:07:43.914681 18929 server_base.cc:1034] running on GCE node
I20260925 22:07:43.914892 18929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:07:43.915143 18929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:07:43.916303 18929 hybrid_clock.cc:648] HybridClock initialized: now 1790374063916279 us; error 55 us; skew 500 ppm
Sep 25 22:07:43 dist-test-slave-7mqr krb5kdc[18822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374063, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:07:43.919036 18929 init.cc:377] Logged in from keytab as kudu/127.5.0.65@KRBTEST.COM (short username kudu)
I20260925 22:07:43.919495 18929 webserver.cc:460] Webserver started at http://127.5.0.65:39111/ using document root <none> and password file <none>
I20260925 22:07:43.919734 18929 fs_manager.cc:359] Metadata directory not provided
I20260925 22:07:43.919778 18929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:07:43.919965 18929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:07:43.921018 18929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/data/instance:
uuid: "ea2086a88bfb40c7afe2181e79e15bf1"
format_stamp: "Formatted at 2026-09-25 22:07:43 on dist-test-slave-7mqr"
server_key: "0e2dd152a80271c53feb36a56c2bb1b7"
server_key_iv: "fbad3187a14e3f4cbbb12307ea495257"
server_key_version: "encryptionkey@0"
I20260925 22:07:43.921379 18929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance:
uuid: "ea2086a88bfb40c7afe2181e79e15bf1"
format_stamp: "Formatted at 2026-09-25 22:07:43 on dist-test-slave-7mqr"
server_key: "0e2dd152a80271c53feb36a56c2bb1b7"
server_key_iv: "fbad3187a14e3f4cbbb12307ea495257"
server_key_version: "encryptionkey@0"
I20260925 22:07:43.922700 18929 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:07:43.923415 18944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:43.923604 18929 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:07:43.923722 18929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/wal
uuid: "ea2086a88bfb40c7afe2181e79e15bf1"
format_stamp: "Formatted at 2026-09-25 22:07:43 on dist-test-slave-7mqr"
server_key: "0e2dd152a80271c53feb36a56c2bb1b7"
server_key_iv: "fbad3187a14e3f4cbbb12307ea495257"
server_key_version: "encryptionkey@0"
I20260925 22:07:43.923804 18929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:07:43.935557 18929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:07:43.937930 18929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:07:43.938040 18929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:07:43.938220 18929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:07:43.938536 18929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:07:43.938570 18929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:43.938613 18929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:07:43.938649 18929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:43.944316 18929 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.65:45619
I20260925 22:07:43.944360 19057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.65:45619 every 8 connection(s)
I20260925 22:07:43.944665 18929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
I20260925 22:07:43.945904 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 18929
I20260925 22:07:43.945997 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance
I20260925 22:07:43.946274 5121 external_mini_cluster.cc:1428] Setting key 2407fb7882285bef15c11c8f46019b9d
Sep 25 22:07:43 dist-test-slave-7mqr krb5kdc[18822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374063, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.0.66@KRBTEST.COM; defaulting to no policy
I20260925 22:07:43.955171 18906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:43.945356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:44227 (local address 127.5.0.126:34019)
0925 22:07:43.945438 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:43.945441 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:43.945635 (+ 194us) server_negotiation.cc:407] Connection header received
0925 22:07:43.946200 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:43.946202 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:43.946215 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:43.946293 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:43.947517 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:43.948374 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:43.949904 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:43.950100 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:43.950155 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:43.952356 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:43.952357 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:43.952358 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:43.952365 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:43.953765 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:43.954202 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:43.954203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:43.954204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:43.954263 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:43.954517 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:43.954518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:43.954518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:43.954637 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:43.954714 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:43.954990 (+ 276us) server_negotiation.cc:299] Negotiation successful
0925 22:07:43.955076 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 22:07:43.955390 19060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:43.945328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.65:44227)
0925 22:07:43.945504 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:43.945545 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:43.946083 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:43.946384 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:43.946388 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:43.947200 (+ 812us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:07:43.947425 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:43.947430 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:43.948483 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:43.948484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:43.949819 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:43.949823 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:43.950185 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:43.950186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:43.950339 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:43.950868 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:07:43.950876 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:07:43.952255 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:07:43.953868 (+ 1613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:43.953875 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:43.953878 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:43.954104 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:43.954333 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:43.954334 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:43.954334 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:43.954443 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:43.954706 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:07:43.954712 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:07:43.954854 (+ 142us) client_negotiation.cc:769] Sending connection context
0925 22:07:43.954958 (+ 104us) client_negotiation.cc:240] Negotiation successful
0925 22:07:43.955112 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":98,"threads_started":1}
I20260925 22:07:43.955971 19058 heartbeater.cc:347] Connected to a master server at 127.5.0.126:34019
I20260925 22:07:43.956102 19058 heartbeater.cc:464] Registering TS with master...
I20260925 22:07:43.956331 19058 heartbeater.cc:511] Master 127.5.0.126:34019 requested a full tablet report, sending...
I20260925 22:07:43.956933 18867 ts_manager.cc:194] Registered new tserver with Master: ea2086a88bfb40c7afe2181e79e15bf1 (127.5.0.65:45619)
I20260925 22:07:43.957638 18867 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.65@KRBTEST.COM'} at 127.5.0.65:44227
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
I20260925 22:07:43.972779 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.66:0
--local_ip_for_outbound_sockets=127.5.0.66
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:34019
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:34219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:07:44.055393 19063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:07:44.055627 19063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:07:44.055667 19063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:07:44.056941 19063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:07:44.057003 19063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:07:44.057054 19063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.66
I20260925 22:07:44.058521 19063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:34219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:34019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.66
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:07:44.058794 19063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:07:44.059036 19063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:07:44.063916 19068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:44.063891 19069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:07:44.063982 19063 server_base.cc:1034] running on GCE node
W20260925 22:07:44.063915 19071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:07:44.064266 19063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:07:44.064519 19063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:07:44.065665 19063 hybrid_clock.cc:648] HybridClock initialized: now 1790374064065645 us; error 36 us; skew 500 ppm
Sep 25 22:07:44 dist-test-slave-7mqr krb5kdc[18822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374064, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:07:44.068084 19063 init.cc:377] Logged in from keytab as kudu/127.5.0.66@KRBTEST.COM (short username kudu)
I20260925 22:07:44.068507 19063 webserver.cc:460] Webserver started at http://127.5.0.66:41497/ using document root <none> and password file <none>
I20260925 22:07:44.068742 19063 fs_manager.cc:359] Metadata directory not provided
I20260925 22:07:44.068826 19063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:07:44.068970 19063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:07:44.069983 19063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/data/instance:
uuid: "6de12817f7c94bdeb7e11241079e9778"
format_stamp: "Formatted at 2026-09-25 22:07:44 on dist-test-slave-7mqr"
server_key: "c1b19e914f4965384d2938c44f636007"
server_key_iv: "a4e7c1b74735afe4fa4291f3d9c0e614"
server_key_version: "encryptionkey@0"
I20260925 22:07:44.070333 19063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance:
uuid: "6de12817f7c94bdeb7e11241079e9778"
format_stamp: "Formatted at 2026-09-25 22:07:44 on dist-test-slave-7mqr"
server_key: "c1b19e914f4965384d2938c44f636007"
server_key_iv: "a4e7c1b74735afe4fa4291f3d9c0e614"
server_key_version: "encryptionkey@0"
I20260925 22:07:44.071588 19063 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:07:44.072337 19078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:44.072502 19063 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:44.072602 19063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/wal
uuid: "6de12817f7c94bdeb7e11241079e9778"
format_stamp: "Formatted at 2026-09-25 22:07:44 on dist-test-slave-7mqr"
server_key: "c1b19e914f4965384d2938c44f636007"
server_key_iv: "a4e7c1b74735afe4fa4291f3d9c0e614"
server_key_version: "encryptionkey@0"
I20260925 22:07:44.072680 19063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:07:44.101653 19063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:07:44.103490 19063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:07:44.103654 19063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:07:44.103909 19063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:07:44.104223 19063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:07:44.104276 19063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:44.104316 19063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:07:44.104362 19063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:44.110051 19063 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.66:44715
I20260925 22:07:44.110100 19191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.66:44715 every 8 connection(s)
I20260925 22:07:44.110399 19063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
Sep 25 22:07:44 dist-test-slave-7mqr krb5kdc[18822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374064, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:07:44.117547 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 19063
I20260925 22:07:44.117653 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance
I20260925 22:07:44.117854 5121 external_mini_cluster.cc:1428] Setting key eb9bb4bb65634f12670312ee65494a2d
I20260925 22:07:44.119208 18906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:44.111117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:43517 (local address 127.5.0.126:34019)
0925 22:07:44.111165 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:44.111168 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:44.111416 (+ 248us) server_negotiation.cc:407] Connection header received
0925 22:07:44.111867 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:44.111868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:44.111877 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:44.111978 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:44.112871 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.113344 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.114270 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.114445 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.114493 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:44.116484 (+ 1991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:44.116486 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:44.116486 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:44.116492 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:44.117833 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:44.118252 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:44.118253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:44.118253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:44.118313 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:44.118561 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:44.118562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:44.118563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:44.118676 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:44.118753 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:44.119023 (+ 270us) server_negotiation.cc:299] Negotiation successful
0925 22:07:44.119111 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:07:44.119357 19194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:44.111122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.66:43517)
0925 22:07:44.111291 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:44.111331 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:44.111789 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:44.112044 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:44.112048 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:44.112609 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:07:44.112795 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:44.112799 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.113419 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.113420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:44.114199 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:44.114201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.114514 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.114515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:44.114622 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:44.115054 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:07:44.115061 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:07:44.116405 (+ 1344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:07:44.117908 (+ 1503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:44.117914 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:44.117916 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:44.118179 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:44.118381 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:44.118381 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:44.118382 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:44.118492 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:44.118744 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:07:44.118750 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:07:44.118885 (+ 135us) client_negotiation.cc:769] Sending connection context
0925 22:07:44.118987 (+ 102us) client_negotiation.cc:240] Negotiation successful
0925 22:07:44.119136 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":81,"threads_started":1}
I20260925 22:07:44.119949 19192 heartbeater.cc:347] Connected to a master server at 127.5.0.126:34019
I20260925 22:07:44.120064 19192 heartbeater.cc:464] Registering TS with master...
I20260925 22:07:44.120231 19192 heartbeater.cc:511] Master 127.5.0.126:34019 requested a full tablet report, sending...
I20260925 22:07:44.120623 18867 ts_manager.cc:194] Registered new tserver with Master: 6de12817f7c94bdeb7e11241079e9778 (127.5.0.66:44715)
I20260925 22:07:44.121105 18867 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.66@KRBTEST.COM'} at 127.5.0.66:43517
WARNING: no policy specified for kudu/127.5.0.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
I20260925 22:07:44.142501 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.67:0
--local_ip_for_outbound_sockets=127.5.0.67
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:34019
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:34219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:07:44.221750 19197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:07:44.221969 19197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:07:44.222005 19197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:07:44.223114 19197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:07:44.223178 19197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:07:44.223224 19197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.67
I20260925 22:07:44.224717 19197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:34219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:34019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.67
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:07:44.224978 19197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:07:44.225203 19197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:07:44.230087 19205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:44.230145 19203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:07:44.230286 19197 server_base.cc:1034] running on GCE node
W20260925 22:07:44.230363 19202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:07:44.230551 19197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:07:44.230737 19197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:07:44.231943 19197 hybrid_clock.cc:648] HybridClock initialized: now 1790374064231937 us; error 34 us; skew 500 ppm
Sep 25 22:07:44 dist-test-slave-7mqr krb5kdc[18822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374064, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:07:44.234264 19197 init.cc:377] Logged in from keytab as kudu/127.5.0.67@KRBTEST.COM (short username kudu)
I20260925 22:07:44.234732 19197 webserver.cc:460] Webserver started at http://127.5.0.67:33617/ using document root <none> and password file <none>
I20260925 22:07:44.234965 19197 fs_manager.cc:359] Metadata directory not provided
I20260925 22:07:44.235034 19197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:07:44.235193 19197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:07:44.236398 19197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/data/instance:
uuid: "dcb501810f564e5eb3dadc544cc81b81"
format_stamp: "Formatted at 2026-09-25 22:07:44 on dist-test-slave-7mqr"
server_key: "4c4f95fcc0c5436faecefcf7ff29589b"
server_key_iv: "34ccab7f0deca77aa655067ee11d3747"
server_key_version: "encryptionkey@0"
I20260925 22:07:44.236763 19197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance:
uuid: "dcb501810f564e5eb3dadc544cc81b81"
format_stamp: "Formatted at 2026-09-25 22:07:44 on dist-test-slave-7mqr"
server_key: "4c4f95fcc0c5436faecefcf7ff29589b"
server_key_iv: "34ccab7f0deca77aa655067ee11d3747"
server_key_version: "encryptionkey@0"
I20260925 22:07:44.238036 19197 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:07:44.238716 19212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:44.238880 19197 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:07:44.238971 19197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/wal
uuid: "dcb501810f564e5eb3dadc544cc81b81"
format_stamp: "Formatted at 2026-09-25 22:07:44 on dist-test-slave-7mqr"
server_key: "4c4f95fcc0c5436faecefcf7ff29589b"
server_key_iv: "34ccab7f0deca77aa655067ee11d3747"
server_key_version: "encryptionkey@0"
I20260925 22:07:44.239053 19197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:07:44.266251 19197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:07:44.268536 19197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:07:44.268708 19197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:07:44.268981 19197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:07:44.269312 19197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:07:44.269366 19197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:44.269402 19197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:07:44.269451 19197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:44.275363 19197 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.67:33761
I20260925 22:07:44.275408 19325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.67:33761 every 8 connection(s)
I20260925 22:07:44.275725 19197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
I20260925 22:07:44.276885 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 19197
I20260925 22:07:44.276960 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance
I20260925 22:07:44.277138 5121 external_mini_cluster.cc:1428] Setting key 6665bfd6eaef694584e4d6ddd50372b1
Sep 25 22:07:44 dist-test-slave-7mqr krb5kdc[18822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374064, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:07:44.285141 18906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:44.276478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:48525 (local address 127.5.0.126:34019)
0925 22:07:44.276523 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:44.276525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:44.276756 (+ 231us) server_negotiation.cc:407] Connection header received
0925 22:07:44.277372 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:44.277374 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:44.277386 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:44.277443 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:44.278621 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.279119 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.280220 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.280421 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.280506 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:44.282493 (+ 1987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:44.282494 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:44.282495 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:44.282502 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:44.283667 (+ 1165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:44.284155 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:44.284156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:44.284156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:44.284215 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:44.284475 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:44.284476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:44.284476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:44.284596 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:44.284672 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:44.284951 (+ 279us) server_negotiation.cc:299] Negotiation successful
0925 22:07:44.285031 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:07:44.285324 19328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:44.276448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.67:48525)
0925 22:07:44.276612 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:44.276653 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:44.277266 (+ 613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:44.277518 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:44.277522 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:44.278291 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:07:44.278498 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:44.278503 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.279227 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.279228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:44.280123 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:44.280125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.280514 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.280516 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:44.280644 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:44.281095 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:07:44.281103 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:07:44.282397 (+ 1294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:07:44.283764 (+ 1367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:44.283770 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:44.283772 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:44.284067 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:44.284289 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:44.284289 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:44.284290 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:44.284402 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:44.284673 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:07:44.284680 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:07:44.284819 (+ 139us) client_negotiation.cc:769] Sending connection context
0925 22:07:44.284920 (+ 101us) client_negotiation.cc:240] Negotiation successful
0925 22:07:44.285081 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":100,"threads_started":1}
I20260925 22:07:44.285835 19326 heartbeater.cc:347] Connected to a master server at 127.5.0.126:34019
I20260925 22:07:44.285928 19326 heartbeater.cc:464] Registering TS with master...
I20260925 22:07:44.286062 19326 heartbeater.cc:511] Master 127.5.0.126:34019 requested a full tablet report, sending...
I20260925 22:07:44.286419 18867 ts_manager.cc:194] Registered new tserver with Master: dcb501810f564e5eb3dadc544cc81b81 (127.5.0.67:33761)
I20260925 22:07:44.286753 18867 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.67@KRBTEST.COM'} at 127.5.0.67:48525
I20260925 22:07:44.288965 5121 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:07:44.295821 18906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:44.290314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38480 (local address 127.5.0.126:34019)
0925 22:07:44.290379 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:44.290381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:44.290444 (+ 63us) server_negotiation.cc:407] Connection header received
0925 22:07:44.290484 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:44.290485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:44.290493 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:44.290534 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:44.291048 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.291506 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.292254 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.292424 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.292474 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:44.293616 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:44.293617 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:44.293617 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:44.293628 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:44.294953 (+ 1325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:44.295229 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:44.295230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:44.295230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:44.295276 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:44.295418 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:44.295418 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:44.295419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:44.295535 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:44.295604 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:44.295685 (+ 81us) server_negotiation.cc:299] Negotiation successful
0925 22:07:44.295738 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 22:07:44.297224 18867 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38480:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:07:44.298729 18867 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:07:44.305408 19341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:44.301363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56524 (local address 127.5.0.67:33761)
0925 22:07:44.301564 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:44.301568 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:44.301581 (+ 13us) server_negotiation.cc:407] Connection header received
0925 22:07:44.301621 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:44.301624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:44.301639 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:44.301780 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:07:44.302137 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.303230 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.304506 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.304985 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.305071 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:44.305129 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:44.305291 (+ 162us) server_negotiation.cc:299] Negotiation successful
0925 22:07:44.305328 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":101,"threads_started":1}
I20260925 22:07:44.305397 19338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:44.301020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:44715 (local address 127.0.0.1:48254)
0925 22:07:44.301171 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:44.301210 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:44.301275 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:44.301909 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:44.301910 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:44.301912 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:07:44.301980 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:44.301982 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.302954 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.302955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:44.304171 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:44.304173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.305118 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.305119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:44.305184 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:44.305189 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:07:44.305226 (+ 37us) client_negotiation.cc:240] Negotiation successful
0925 22:07:44.305281 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":46,"threads_started":1}
I20260925 22:07:44.305542 19339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:44.301030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:33761 (local address 127.0.0.1:56524)
0925 22:07:44.301487 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:44.301500 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:44.301555 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:44.301873 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:44.301877 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:44.301879 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:07:44.302055 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:44.302057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.303340 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.303341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:44.304419 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:44.304420 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.305072 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.305073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:44.305161 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:44.305174 (+ 13us) client_negotiation.cc:769] Sending connection context
0925 22:07:44.305273 (+ 99us) client_negotiation.cc:240] Negotiation successful
0925 22:07:44.305315 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"mutex_wait_us":8,"thread_start_us":76,"threads_started":1}
I20260925 22:07:44.305761 19340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:44.301329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48818 (local address 127.5.0.65:45619)
0925 22:07:44.301509 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:44.301515 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:44.301826 (+ 311us) server_negotiation.cc:407] Connection header received
0925 22:07:44.301951 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:44.301953 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:44.301969 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:44.302023 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:07:44.302225 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.303286 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.304608 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.305299 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.305340 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:44.305391 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:44.305649 (+ 258us) server_negotiation.cc:299] Negotiation successful
0925 22:07:44.305688 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":72,"threads_started":1}
I20260925 22:07:44.305948 19337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:44.301210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:45619 (local address 127.0.0.1:48818)
0925 22:07:44.301758 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:44.301768 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:44.301854 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:44.302124 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:44.302124 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:44.302126 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:07:44.302158 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:44.302159 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.303387 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.303388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:44.304498 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:44.304501 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.305450 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.305451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:44.305545 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:44.305549 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:07:44.305843 (+ 294us) client_negotiation.cc:240] Negotiation successful
0925 22:07:44.305885 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":80,"threads_started":1}
I20260925 22:07:44.306262 19260 tablet_service.cc:1467] Processing CreateTablet for tablet ccdbabb1fb1f43c692d428ccfa77fdbf (DEFAULT_TABLE table=test-table [id=c0ca3c7804724cc5b67cc1881eb24658]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:07:44.306453 19342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:44.301575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48254 (local address 127.5.0.66:44715)
0925 22:07:44.301731 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:44.301736 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:44.301747 (+ 11us) server_negotiation.cc:407] Connection header received
0925 22:07:44.301790 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:44.301792 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:44.301808 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:44.302060 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:07:44.302146 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.302845 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.304256 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.305008 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.305366 (+ 358us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:44.305427 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:44.306313 (+ 886us) server_negotiation.cc:299] Negotiation successful
0925 22:07:44.306358 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":96,"threads_started":1}
I20260925 22:07:44.306659 19260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccdbabb1fb1f43c692d428ccfa77fdbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:07:44.306653 18992 tablet_service.cc:1467] Processing CreateTablet for tablet ccdbabb1fb1f43c692d428ccfa77fdbf (DEFAULT_TABLE table=test-table [id=c0ca3c7804724cc5b67cc1881eb24658]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:07:44.306967 18992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccdbabb1fb1f43c692d428ccfa77fdbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:07:44.307134 19126 tablet_service.cc:1467] Processing CreateTablet for tablet ccdbabb1fb1f43c692d428ccfa77fdbf (DEFAULT_TABLE table=test-table [id=c0ca3c7804724cc5b67cc1881eb24658]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:07:44.307407 19126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccdbabb1fb1f43c692d428ccfa77fdbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:07:44.309303 19343 tablet_bootstrap.cc:492] T ccdbabb1fb1f43c692d428ccfa77fdbf P dcb501810f564e5eb3dadc544cc81b81: Bootstrap starting.
I20260925 22:07:44.309882 19343 tablet_bootstrap.cc:654] T ccdbabb1fb1f43c692d428ccfa77fdbf P dcb501810f564e5eb3dadc544cc81b81: Neither blocks nor log segments found. Creating new log.
I20260925 22:07:44.310268 19344 tablet_bootstrap.cc:492] T ccdbabb1fb1f43c692d428ccfa77fdbf P ea2086a88bfb40c7afe2181e79e15bf1: Bootstrap starting.
I20260925 22:07:44.310513 19345 tablet_bootstrap.cc:492] T ccdbabb1fb1f43c692d428ccfa77fdbf P 6de12817f7c94bdeb7e11241079e9778: Bootstrap starting.
I20260925 22:07:44.310582 19343 log.cc:824] T ccdbabb1fb1f43c692d428ccfa77fdbf P dcb501810f564e5eb3dadc544cc81b81: Log is configured to *not* fsync() on all Append() calls
I20260925 22:07:44.311017 19344 tablet_bootstrap.cc:654] T ccdbabb1fb1f43c692d428ccfa77fdbf P ea2086a88bfb40c7afe2181e79e15bf1: Neither blocks nor log segments found. Creating new log.
I20260925 22:07:44.311196 19345 tablet_bootstrap.cc:654] T ccdbabb1fb1f43c692d428ccfa77fdbf P 6de12817f7c94bdeb7e11241079e9778: Neither blocks nor log segments found. Creating new log.
I20260925 22:07:44.311574 19344 log.cc:824] T ccdbabb1fb1f43c692d428ccfa77fdbf P ea2086a88bfb40c7afe2181e79e15bf1: Log is configured to *not* fsync() on all Append() calls
I20260925 22:07:44.311685 19343 tablet_bootstrap.cc:492] T ccdbabb1fb1f43c692d428ccfa77fdbf P dcb501810f564e5eb3dadc544cc81b81: No bootstrap required, opened a new log
I20260925 22:07:44.311715 19345 log.cc:824] T ccdbabb1fb1f43c692d428ccfa77fdbf P 6de12817f7c94bdeb7e11241079e9778: Log is configured to *not* fsync() on all Append() calls
I20260925 22:07:44.311755 19343 ts_tablet_manager.cc:1397] T ccdbabb1fb1f43c692d428ccfa77fdbf P dcb501810f564e5eb3dadc544cc81b81: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260925 22:07:44.312391 19344 tablet_bootstrap.cc:492] T ccdbabb1fb1f43c692d428ccfa77fdbf P ea2086a88bfb40c7afe2181e79e15bf1: No bootstrap required, opened a new log
I20260925 22:07:44.312439 19345 tablet_bootstrap.cc:492] T ccdbabb1fb1f43c692d428ccfa77fdbf P 6de12817f7c94bdeb7e11241079e9778: No bootstrap required, opened a new log
I20260925 22:07:44.312466 19344 ts_tablet_manager.cc:1397] T ccdbabb1fb1f43c692d428ccfa77fdbf P ea2086a88bfb40c7afe2181e79e15bf1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:07:44.312489 19345 ts_tablet_manager.cc:1397] T ccdbabb1fb1f43c692d428ccfa77fdbf P 6de12817f7c94bdeb7e11241079e9778: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 22:07:44.313303 19343 raft_consensus.cc:348] T ccdbabb1fb1f43c692d428ccfa77fdbf P dcb501810f564e5eb3dadc544cc81b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6de12817f7c94bdeb7e11241079e9778" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44715 } } peers { permanent_uuid: "dcb501810f564e5eb3dadc544cc81b81" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33761 } } peers { permanent_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45619 } }
I20260925 22:07:44.313459 19343 raft_consensus.cc:374] T ccdbabb1fb1f43c692d428ccfa77fdbf P dcb501810f564e5eb3dadc544cc81b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:07:44.313503 19343 raft_consensus.cc:729] T ccdbabb1fb1f43c692d428ccfa77fdbf P dcb501810f564e5eb3dadc544cc81b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcb501810f564e5eb3dadc544cc81b81, State: Initialized, Role: FOLLOWER
I20260925 22:07:44.313640 19343 consensus_queue.cc:260] T ccdbabb1fb1f43c692d428ccfa77fdbf P dcb501810f564e5eb3dadc544cc81b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6de12817f7c94bdeb7e11241079e9778" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44715 } } peers { permanent_uuid: "dcb501810f564e5eb3dadc544cc81b81" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33761 } } peers { permanent_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45619 } }
I20260925 22:07:44.313900 19343 ts_tablet_manager.cc:1428] T ccdbabb1fb1f43c692d428ccfa77fdbf P dcb501810f564e5eb3dadc544cc81b81: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:07:44.314040 19326 heartbeater.cc:503] Master 127.5.0.126:34019 was elected leader, sending a full tablet report...
I20260925 22:07:44.314395 19344 raft_consensus.cc:348] T ccdbabb1fb1f43c692d428ccfa77fdbf P ea2086a88bfb40c7afe2181e79e15bf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6de12817f7c94bdeb7e11241079e9778" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44715 } } peers { permanent_uuid: "dcb501810f564e5eb3dadc544cc81b81" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33761 } } peers { permanent_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45619 } }
I20260925 22:07:44.314599 19344 raft_consensus.cc:374] T ccdbabb1fb1f43c692d428ccfa77fdbf P ea2086a88bfb40c7afe2181e79e15bf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:07:44.314636 19344 raft_consensus.cc:729] T ccdbabb1fb1f43c692d428ccfa77fdbf P ea2086a88bfb40c7afe2181e79e15bf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea2086a88bfb40c7afe2181e79e15bf1, State: Initialized, Role: FOLLOWER
I20260925 22:07:44.314756 19344 consensus_queue.cc:260] T ccdbabb1fb1f43c692d428ccfa77fdbf P ea2086a88bfb40c7afe2181e79e15bf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6de12817f7c94bdeb7e11241079e9778" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44715 } } peers { permanent_uuid: "dcb501810f564e5eb3dadc544cc81b81" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33761 } } peers { permanent_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45619 } }
I20260925 22:07:44.314704 19345 raft_consensus.cc:348] T ccdbabb1fb1f43c692d428ccfa77fdbf P 6de12817f7c94bdeb7e11241079e9778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6de12817f7c94bdeb7e11241079e9778" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44715 } } peers { permanent_uuid: "dcb501810f564e5eb3dadc544cc81b81" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33761 } } peers { permanent_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45619 } }
I20260925 22:07:44.314852 19345 raft_consensus.cc:374] T ccdbabb1fb1f43c692d428ccfa77fdbf P 6de12817f7c94bdeb7e11241079e9778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:07:44.314903 19345 raft_consensus.cc:729] T ccdbabb1fb1f43c692d428ccfa77fdbf P 6de12817f7c94bdeb7e11241079e9778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6de12817f7c94bdeb7e11241079e9778, State: Initialized, Role: FOLLOWER
I20260925 22:07:44.315040 19344 ts_tablet_manager.cc:1428] T ccdbabb1fb1f43c692d428ccfa77fdbf P ea2086a88bfb40c7afe2181e79e15bf1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 22:07:44.315048 19345 consensus_queue.cc:260] T ccdbabb1fb1f43c692d428ccfa77fdbf P 6de12817f7c94bdeb7e11241079e9778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6de12817f7c94bdeb7e11241079e9778" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44715 } } peers { permanent_uuid: "dcb501810f564e5eb3dadc544cc81b81" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33761 } } peers { permanent_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45619 } }
I20260925 22:07:44.315156 19058 heartbeater.cc:503] Master 127.5.0.126:34019 was elected leader, sending a full tablet report...
I20260925 22:07:44.315304 19345 ts_tablet_manager.cc:1428] T ccdbabb1fb1f43c692d428ccfa77fdbf P 6de12817f7c94bdeb7e11241079e9778: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260925 22:07:44.315429 19192 heartbeater.cc:503] Master 127.5.0.126:34019 was elected leader, sending a full tablet report...
W20260925 22:07:44.360970 19193 tablet.cc:2369] T ccdbabb1fb1f43c692d428ccfa77fdbf P 6de12817f7c94bdeb7e11241079e9778: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 22:07:44.445340 19059 tablet.cc:2369] T ccdbabb1fb1f43c692d428ccfa77fdbf P ea2086a88bfb40c7afe2181e79e15bf1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 22:07:44.526275 19327 tablet.cc:2369] T ccdbabb1fb1f43c692d428ccfa77fdbf P dcb501810f564e5eb3dadc544cc81b81: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:07:44.569247 19349 raft_consensus.cc:482] T ccdbabb1fb1f43c692d428ccfa77fdbf P dcb501810f564e5eb3dadc544cc81b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:07:44.569376 19349 raft_consensus.cc:504] T ccdbabb1fb1f43c692d428ccfa77fdbf P dcb501810f564e5eb3dadc544cc81b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6de12817f7c94bdeb7e11241079e9778" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44715 } } peers { permanent_uuid: "dcb501810f564e5eb3dadc544cc81b81" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33761 } } peers { permanent_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45619 } }
I20260925 22:07:44.569818 19349 leader_election.cc:288] T ccdbabb1fb1f43c692d428ccfa77fdbf P dcb501810f564e5eb3dadc544cc81b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6de12817f7c94bdeb7e11241079e9778 (127.5.0.66:44715), ea2086a88bfb40c7afe2181e79e15bf1 (127.5.0.65:45619)
I20260925 22:07:44.573768 19342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:44.570127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:48699 (local address 127.5.0.66:44715)
0925 22:07:44.570196 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:44.570198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:44.570210 (+ 12us) server_negotiation.cc:407] Connection header received
0925 22:07:44.570242 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:44.570243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:44.570255 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:44.570347 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:07:44.570515 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.571296 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.572819 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.573365 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.573439 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:44.573450 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:44.573661 (+ 211us) server_negotiation.cc:299] Negotiation successful
0925 22:07:44.573697 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 22:07:44.573769 19340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:44.570283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:45039 (local address 127.5.0.65:45619)
0925 22:07:44.570350 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:44.570353 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:44.570438 (+ 85us) server_negotiation.cc:407] Connection header received
0925 22:07:44.570501 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:44.570502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:44.570514 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:44.570571 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:07:44.570704 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.571381 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.572819 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.573362 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.573439 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:44.573450 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:44.573661 (+ 211us) server_negotiation.cc:299] Negotiation successful
0925 22:07:44.573700 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 22:07:44.573788 19328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:44.570048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:44715 (local address 127.5.0.67:48699)
0925 22:07:44.570112 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:44.570125 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:44.570173 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:44.570361 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:44.570362 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:44.570363 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:07:44.570438 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:44.570440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.571408 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.571409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:44.572722 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:44.572725 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.573482 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.573483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:44.573574 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:44.573576 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 22:07:44.573635 (+ 59us) client_negotiation.cc:240] Negotiation successful
0925 22:07:44.573683 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":7}
I20260925 22:07:44.573794 19352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:44.570199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:45619 (local address 127.5.0.67:45039)
0925 22:07:44.570364 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:44.570375 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:44.570429 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:44.570586 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:44.570587 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:44.570588 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:07:44.570636 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:44.570637 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.571493 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.571494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:44.572722 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:44.572725 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.573482 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.573483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:44.573565 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:44.573570 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:07:44.573635 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 22:07:44.573685 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":26,"thread_start_us":105,"threads_started":1}
I20260925 22:07:44.574195 19146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccdbabb1fb1f43c692d428ccfa77fdbf" candidate_uuid: "dcb501810f564e5eb3dadc544cc81b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6de12817f7c94bdeb7e11241079e9778" is_pre_election: true
I20260925 22:07:44.574223 19012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccdbabb1fb1f43c692d428ccfa77fdbf" candidate_uuid: "dcb501810f564e5eb3dadc544cc81b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" is_pre_election: true
I20260925 22:07:44.574334 19146 raft_consensus.cc:2454] T ccdbabb1fb1f43c692d428ccfa77fdbf P 6de12817f7c94bdeb7e11241079e9778 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dcb501810f564e5eb3dadc544cc81b81 in term 0.
I20260925 22:07:44.574393 19012 raft_consensus.cc:2454] T ccdbabb1fb1f43c692d428ccfa77fdbf P ea2086a88bfb40c7afe2181e79e15bf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dcb501810f564e5eb3dadc544cc81b81 in term 0.
I20260925 22:07:44.574595 19215 leader_election.cc:302] T ccdbabb1fb1f43c692d428ccfa77fdbf P dcb501810f564e5eb3dadc544cc81b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6de12817f7c94bdeb7e11241079e9778, dcb501810f564e5eb3dadc544cc81b81; no voters:
I20260925 22:07:44.574782 19349 raft_consensus.cc:2781] T ccdbabb1fb1f43c692d428ccfa77fdbf P dcb501810f564e5eb3dadc544cc81b81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:07:44.574858 19349 raft_consensus.cc:482] T ccdbabb1fb1f43c692d428ccfa77fdbf P dcb501810f564e5eb3dadc544cc81b81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:07:44.574877 19349 raft_consensus.cc:3037] T ccdbabb1fb1f43c692d428ccfa77fdbf P dcb501810f564e5eb3dadc544cc81b81 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:07:44.575711 19349 raft_consensus.cc:504] T ccdbabb1fb1f43c692d428ccfa77fdbf P dcb501810f564e5eb3dadc544cc81b81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6de12817f7c94bdeb7e11241079e9778" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44715 } } peers { permanent_uuid: "dcb501810f564e5eb3dadc544cc81b81" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33761 } } peers { permanent_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45619 } }
I20260925 22:07:44.575852 19349 leader_election.cc:288] T ccdbabb1fb1f43c692d428ccfa77fdbf P dcb501810f564e5eb3dadc544cc81b81 [CANDIDATE]: Term 1 election: Requested vote from peers 6de12817f7c94bdeb7e11241079e9778 (127.5.0.66:44715), ea2086a88bfb40c7afe2181e79e15bf1 (127.5.0.65:45619)
I20260925 22:07:44.576066 19146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccdbabb1fb1f43c692d428ccfa77fdbf" candidate_uuid: "dcb501810f564e5eb3dadc544cc81b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6de12817f7c94bdeb7e11241079e9778"
I20260925 22:07:44.576153 19146 raft_consensus.cc:3037] T ccdbabb1fb1f43c692d428ccfa77fdbf P 6de12817f7c94bdeb7e11241079e9778 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:07:44.576187 19012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccdbabb1fb1f43c692d428ccfa77fdbf" candidate_uuid: "dcb501810f564e5eb3dadc544cc81b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea2086a88bfb40c7afe2181e79e15bf1"
I20260925 22:07:44.576274 19012 raft_consensus.cc:3037] T ccdbabb1fb1f43c692d428ccfa77fdbf P ea2086a88bfb40c7afe2181e79e15bf1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:07:44.576932 19146 raft_consensus.cc:2454] T ccdbabb1fb1f43c692d428ccfa77fdbf P 6de12817f7c94bdeb7e11241079e9778 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcb501810f564e5eb3dadc544cc81b81 in term 1.
I20260925 22:07:44.577044 19012 raft_consensus.cc:2454] T ccdbabb1fb1f43c692d428ccfa77fdbf P ea2086a88bfb40c7afe2181e79e15bf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcb501810f564e5eb3dadc544cc81b81 in term 1.
I20260925 22:07:44.577105 19215 leader_election.cc:302] T ccdbabb1fb1f43c692d428ccfa77fdbf P dcb501810f564e5eb3dadc544cc81b81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6de12817f7c94bdeb7e11241079e9778, dcb501810f564e5eb3dadc544cc81b81; no voters:
I20260925 22:07:44.577224 19349 raft_consensus.cc:2781] T ccdbabb1fb1f43c692d428ccfa77fdbf P dcb501810f564e5eb3dadc544cc81b81 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:07:44.577374 19349 raft_consensus.cc:686] T ccdbabb1fb1f43c692d428ccfa77fdbf P dcb501810f564e5eb3dadc544cc81b81 [term 1 LEADER]: Becoming Leader. State: Replica: dcb501810f564e5eb3dadc544cc81b81, State: Running, Role: LEADER
I20260925 22:07:44.577476 19349 consensus_queue.cc:237] T ccdbabb1fb1f43c692d428ccfa77fdbf P dcb501810f564e5eb3dadc544cc81b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6de12817f7c94bdeb7e11241079e9778" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44715 } } peers { permanent_uuid: "dcb501810f564e5eb3dadc544cc81b81" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33761 } } peers { permanent_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45619 } }
I20260925 22:07:44.578352 18866 catalog_manager.cc:5495] T ccdbabb1fb1f43c692d428ccfa77fdbf P dcb501810f564e5eb3dadc544cc81b81 reported cstate change: term changed from 0 to 1, leader changed from <none> to dcb501810f564e5eb3dadc544cc81b81 (127.5.0.67). New cstate: current_term: 1 leader_uuid: "dcb501810f564e5eb3dadc544cc81b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6de12817f7c94bdeb7e11241079e9778" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dcb501810f564e5eb3dadc544cc81b81" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45619 } health_report { overall_health: UNKNOWN } } }
I20260925 22:07:44.587208 19341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:44.585158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56534 (local address 127.5.0.67:33761)
0925 22:07:44.585214 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:44.585216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:44.585247 (+ 31us) server_negotiation.cc:407] Connection header received
0925 22:07:44.585279 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:44.585280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:44.585295 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:44.585345 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:07:44.585454 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.585907 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.586615 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.586790 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.586837 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:44.586953 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:07:44.587040 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:07:44.587097 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:44.587129 (+ 32us) server_negotiation.cc:299] Negotiation successful
0925 22:07:44.587144 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 22:07:44.588840 19146 raft_consensus.cc:1261] T ccdbabb1fb1f43c692d428ccfa77fdbf P 6de12817f7c94bdeb7e11241079e9778 [term 1 FOLLOWER]: Refusing update from remote peer dcb501810f564e5eb3dadc544cc81b81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:07:44.588896 19012 raft_consensus.cc:1261] T ccdbabb1fb1f43c692d428ccfa77fdbf P ea2086a88bfb40c7afe2181e79e15bf1 [term 1 FOLLOWER]: Refusing update from remote peer dcb501810f564e5eb3dadc544cc81b81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:07:44.589198 19349 consensus_queue.cc:1035] T ccdbabb1fb1f43c692d428ccfa77fdbf P dcb501810f564e5eb3dadc544cc81b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:07:44.589313 19353 consensus_queue.cc:1035] T ccdbabb1fb1f43c692d428ccfa77fdbf P dcb501810f564e5eb3dadc544cc81b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6de12817f7c94bdeb7e11241079e9778" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:07:44.591391 19356 mvcc.cc:205] Tried to move back new op lower bound from 7333372168553553920 to 7333372168510009344. Current Snapshot: MvccSnapshot[applied={T|T < 7333372168553553920}]
I20260925 22:07:44.591565 19358 mvcc.cc:205] Tried to move back new op lower bound from 7333372168553553920 to 7333372168510009344. Current Snapshot: MvccSnapshot[applied={T|T < 7333372168553553920}]
I20260925 22:07:44.592053 19357 mvcc.cc:205] Tried to move back new op lower bound from 7333372168553553920 to 7333372168510009344. Current Snapshot: MvccSnapshot[applied={T|T < 7333372168553553920 or (T in {7333372168553553920})}]
I20260925 22:07:44.594121 18867 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:07:44.594216 18867 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:07:44.594924 18867 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572: Sending DeleteTablet for 3 replicas of tablet ccdbabb1fb1f43c692d428ccfa77fdbf
I20260925 22:07:44.595239 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 18929
I20260925 22:07:44.595243 19260 tablet_service.cc:1514] Processing DeleteTablet for tablet ccdbabb1fb1f43c692d428ccfa77fdbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:07:44 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:56524
I20260925 22:07:44.595291 19126 tablet_service.cc:1514] Processing DeleteTablet for tablet ccdbabb1fb1f43c692d428ccfa77fdbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:07:44 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:48254
I20260925 22:07:44.600044 19367 tablet_replica.cc:330] T ccdbabb1fb1f43c692d428ccfa77fdbf P 6de12817f7c94bdeb7e11241079e9778: stopping tablet replica
I20260925 22:07:44.600165 19367 raft_consensus.cc:2229] T ccdbabb1fb1f43c692d428ccfa77fdbf P 6de12817f7c94bdeb7e11241079e9778 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:07:44.600272 19366 tablet_replica.cc:330] T ccdbabb1fb1f43c692d428ccfa77fdbf P dcb501810f564e5eb3dadc544cc81b81: stopping tablet replica
I20260925 22:07:44.600284 19367 raft_consensus.cc:2258] T ccdbabb1fb1f43c692d428ccfa77fdbf P 6de12817f7c94bdeb7e11241079e9778 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:07:44.600411 19366 raft_consensus.cc:2229] T ccdbabb1fb1f43c692d428ccfa77fdbf P dcb501810f564e5eb3dadc544cc81b81 [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:07:44.600572 19366 raft_consensus.cc:2258] T ccdbabb1fb1f43c692d428ccfa77fdbf P dcb501810f564e5eb3dadc544cc81b81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:07:44.600898 19367 ts_tablet_manager.cc:1905] T ccdbabb1fb1f43c692d428ccfa77fdbf P 6de12817f7c94bdeb7e11241079e9778: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:07:44.600939 19366 ts_tablet_manager.cc:1905] T ccdbabb1fb1f43c692d428ccfa77fdbf P dcb501810f564e5eb3dadc544cc81b81: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 22:07:44.602134 18853 connection.cc:535] client connection to 127.5.0.65:45619 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 22:07:44.602228 18853 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 22:07:44.602612 19367 ts_tablet_manager.cc:1918] T ccdbabb1fb1f43c692d428ccfa77fdbf P 6de12817f7c94bdeb7e11241079e9778: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:07:44.602699 19367 log.cc:1196] T ccdbabb1fb1f43c692d428ccfa77fdbf P 6de12817f7c94bdeb7e11241079e9778: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/wal/wals/ccdbabb1fb1f43c692d428ccfa77fdbf
I20260925 22:07:44.602717 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 19063
I20260925 22:07:44.602962 19367 ts_tablet_manager.cc:1939] T ccdbabb1fb1f43c692d428ccfa77fdbf P 6de12817f7c94bdeb7e11241079e9778: Deleting consensus metadata
I20260925 22:07:44.602950 19366 ts_tablet_manager.cc:1918] T ccdbabb1fb1f43c692d428ccfa77fdbf P dcb501810f564e5eb3dadc544cc81b81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:07:44.603103 19366 log.cc:1196] T ccdbabb1fb1f43c692d428ccfa77fdbf P dcb501810f564e5eb3dadc544cc81b81: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/wal/wals/ccdbabb1fb1f43c692d428ccfa77fdbf
I20260925 22:07:44.603448 19366 ts_tablet_manager.cc:1939] T ccdbabb1fb1f43c692d428ccfa77fdbf P dcb501810f564e5eb3dadc544cc81b81: Deleting consensus metadata
I20260925 22:07:44.605271 18851 catalog_manager.cc:4841] TS dcb501810f564e5eb3dadc544cc81b81 (127.5.0.67:33761): tablet ccdbabb1fb1f43c692d428ccfa77fdbf (table test-table [id=c0ca3c7804724cc5b67cc1881eb24658]) successfully deleted
I20260925 22:07:44.605458 19337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:44.605331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:45619 (local address 127.0.0.1:48822)
0925 22:07:44.605382 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:44.605422 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.65:45619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260925 22:07:44.605598 18853 catalog_manager.cc:4584] TS ea2086a88bfb40c7afe2181e79e15bf1 (127.5.0.65:45619): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ccdbabb1fb1f43c692d428ccfa77fdbf: Network error: Client connection negotiation failed: client connection to 127.5.0.65:45619: connect: Connection refused (error 111)
I20260925 22:07:44.607765 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 19197
I20260925 22:07:44.607824 19337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:44.607692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:44715 (local address 127.0.0.1:48258)
0925 22:07:44.607763 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:44.607793 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.66:44715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260925 22:07:44.612535 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 18835
I20260925 22:07:44.617890 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:34019
--webserver_interface=127.5.0.126
--webserver_port=44265
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:34219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:34019
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:07:44.698259 19369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:07:44.698494 19369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:07:44.698518 19369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:07:44.699712 19369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:07:44.699756 19369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:07:44.699771 19369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:07:44.699808 19369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:07:44.699839 19369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:07:44.701370 19369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:34219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:34019
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:34019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=44265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:07:44.701640 19369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:07:44.701881 19369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:07:44.710547 19378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:44.710568 19375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:44.710624 19376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:07:44.710806 19369 server_base.cc:1034] running on GCE node
I20260925 22:07:44.711072 19369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:07:44.711421 19369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:07:44.712569 19369 hybrid_clock.cc:648] HybridClock initialized: now 1790374064712548 us; error 40 us; skew 500 ppm
Sep 25 22:07:44 dist-test-slave-7mqr krb5kdc[18822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374064, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:07:44.715714 19369 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:07:44.716224 19369 webserver.cc:460] Webserver started at http://127.5.0.126:44265/ using document root <none> and password file <none>
I20260925 22:07:44.716439 19369 fs_manager.cc:359] Metadata directory not provided
I20260925 22:07:44.716490 19369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:07:44.717804 19369 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:07:44.718374 19385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:44.718533 19369 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:07:44.718649 19369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "475894d649af404baa1eb2718c7b6572"
format_stamp: "Formatted at 2026-09-25 22:07:43 on dist-test-slave-7mqr"
server_key: "630a3b972c59128422a37a0c489e9282"
server_key_iv: "36ca75de1ef56b8114a0b21502ee1d11"
server_key_version: "encryptionkey@0"
I20260925 22:07:44.719051 19369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:07:44.739857 19369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:07:44.741907 19369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:07:44.742074 19369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:07:44.746078 19369 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:34019
I20260925 22:07:44.746114 19437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:34019 every 8 connection(s)
I20260925 22:07:44.746541 19369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:07:44.748440 19438 sys_catalog.cc:263] Verifying existing consensus state
I20260925 22:07:44.749095 19438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572: Bootstrap starting.
I20260925 22:07:44.752895 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 19369
I20260925 22:07:44.753188 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.65:45619
--local_ip_for_outbound_sockets=127.5.0.65
--tserver_master_addrs=127.5.0.126:34019
--webserver_port=39111
--webserver_interface=127.5.0.65
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:34219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:07:44.757835 19438 log.cc:824] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572: Log is configured to *not* fsync() on all Append() calls
I20260925 22:07:44.759778 19438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 22:07:44.760118 19438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572: Bootstrap complete.
I20260925 22:07:44.762533 19438 raft_consensus.cc:348] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "475894d649af404baa1eb2718c7b6572" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34019 } }
I20260925 22:07:44.762774 19438 raft_consensus.cc:729] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 475894d649af404baa1eb2718c7b6572, State: Initialized, Role: FOLLOWER
I20260925 22:07:44.762941 19438 consensus_queue.cc:260] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "475894d649af404baa1eb2718c7b6572" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34019 } }
I20260925 22:07:44.763010 19438 raft_consensus.cc:388] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:07:44.763078 19438 raft_consensus.cc:482] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:07:44.763126 19438 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [term 1 FOLLOWER]: Advancing to term 2
I20260925 22:07:44.764272 19438 raft_consensus.cc:504] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "475894d649af404baa1eb2718c7b6572" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34019 } }
I20260925 22:07:44.764389 19438 leader_election.cc:302] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 475894d649af404baa1eb2718c7b6572; no voters:
I20260925 22:07:44.764619 19438 leader_election.cc:288] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 22:07:44.764667 19442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 22:07:44.764784 19442 raft_consensus.cc:686] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [term 2 LEADER]: Becoming Leader. State: Replica: 475894d649af404baa1eb2718c7b6572, State: Running, Role: LEADER
I20260925 22:07:44.764899 19438 sys_catalog.cc:564] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:07:44.764875 19442 consensus_queue.cc:237] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "475894d649af404baa1eb2718c7b6572" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34019 } }
I20260925 22:07:44.765247 19443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "475894d649af404baa1eb2718c7b6572" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "475894d649af404baa1eb2718c7b6572" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34019 } } }
I20260925 22:07:44.765261 19444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 475894d649af404baa1eb2718c7b6572. Latest consensus state: current_term: 2 leader_uuid: "475894d649af404baa1eb2718c7b6572" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "475894d649af404baa1eb2718c7b6572" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34019 } } }
I20260925 22:07:44.765411 19443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [sys.catalog]: This master's current role is: LEADER
I20260925 22:07:44.765432 19444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [sys.catalog]: This master's current role is: LEADER
I20260925 22:07:44.766038 19453 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:07:44.766865 19453 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:07:44.766986 19453 catalog_manager.cc:1259] Loaded cluster ID: 3fc71eb117d04324bef4c9c870cd2682
I20260925 22:07:44.767021 19453 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:07:44.767455 19453 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260925 22:07:44.772568 19453 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260925 22:07:44.774964 19453 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:07:44.775164 19453 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572: Loaded TSK: 0
I20260925 22:07:44.775373 19453 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 22:07:44.839192 19440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:07:44.839399 19440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:07:44.839423 19440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:07:44.840653 19440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:07:44.840696 19440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:07:44.840723 19440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.65
I20260925 22:07:44.842150 19440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:34219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.65:45619
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.0.65
--webserver_port=39111
--tserver_master_addrs=127.5.0.126:34019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.65
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:07:44.842379 19440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:07:44.842619 19440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:07:44.847499 19466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:44.847508 19465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:44.847600 19468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:07:44.847966 19440 server_base.cc:1034] running on GCE node
I20260925 22:07:44.848186 19440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:07:44.848423 19440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:07:44.849582 19440 hybrid_clock.cc:648] HybridClock initialized: now 1790374064849563 us; error 37 us; skew 500 ppm
Sep 25 22:07:44 dist-test-slave-7mqr krb5kdc[18822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374064, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:07:44.852207 19440 init.cc:377] Logged in from keytab as kudu/127.5.0.65@KRBTEST.COM (short username kudu)
I20260925 22:07:44.852718 19440 webserver.cc:460] Webserver started at http://127.5.0.65:39111/ using document root <none> and password file <none>
I20260925 22:07:44.852947 19440 fs_manager.cc:359] Metadata directory not provided
I20260925 22:07:44.853026 19440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:07:44.854413 19440 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:07:44.855123 19475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:44.855279 19440 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:07:44.855381 19440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/wal
uuid: "ea2086a88bfb40c7afe2181e79e15bf1"
format_stamp: "Formatted at 2026-09-25 22:07:43 on dist-test-slave-7mqr"
server_key: "0e2dd152a80271c53feb36a56c2bb1b7"
server_key_iv: "fbad3187a14e3f4cbbb12307ea495257"
server_key_version: "encryptionkey@0"
I20260925 22:07:44.855719 19440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:07:44.866072 19440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:07:44.868270 19440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:07:44.868412 19440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:07:44.868664 19440 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:07:44.869113 19482 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 22:07:44.870584 19440 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 22:07:44.870628 19440 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260925 22:07:44.870653 19440 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 22:07:44.871313 19440 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 22:07:44.871356 19440 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260925 22:07:44.871399 19482 tablet_bootstrap.cc:492] T ccdbabb1fb1f43c692d428ccfa77fdbf P ea2086a88bfb40c7afe2181e79e15bf1: Bootstrap starting.
I20260925 22:07:44.879288 19440 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.65:45619
I20260925 22:07:44.879467 19589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.65:45619 every 8 connection(s)
I20260925 22:07:44.879828 19440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
I20260925 22:07:44.882952 19482 log.cc:824] T ccdbabb1fb1f43c692d428ccfa77fdbf P ea2086a88bfb40c7afe2181e79e15bf1: Log is configured to *not* fsync() on all Append() calls
I20260925 22:07:44.885807 19482 tablet_bootstrap.cc:492] T ccdbabb1fb1f43c692d428ccfa77fdbf P ea2086a88bfb40c7afe2181e79e15bf1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 22:07:44.886139 19482 tablet_bootstrap.cc:492] T ccdbabb1fb1f43c692d428ccfa77fdbf P ea2086a88bfb40c7afe2181e79e15bf1: Bootstrap complete.
I20260925 22:07:44.887357 19482 ts_tablet_manager.cc:1397] T ccdbabb1fb1f43c692d428ccfa77fdbf P ea2086a88bfb40c7afe2181e79e15bf1: Time spent bootstrapping tablet: real 0.016s user 0.012s sys 0.000s
I20260925 22:07:44.888613 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 19440
I20260925 22:07:44.888888 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.66:44715
--local_ip_for_outbound_sockets=127.5.0.66
--tserver_master_addrs=127.5.0.126:34019
--webserver_port=41497
--webserver_interface=127.5.0.66
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:34219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:07:44.888618 19482 raft_consensus.cc:348] T ccdbabb1fb1f43c692d428ccfa77fdbf P ea2086a88bfb40c7afe2181e79e15bf1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6de12817f7c94bdeb7e11241079e9778" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44715 } } peers { permanent_uuid: "dcb501810f564e5eb3dadc544cc81b81" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33761 } } peers { permanent_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45619 } }
I20260925 22:07:44.888958 19482 raft_consensus.cc:729] T ccdbabb1fb1f43c692d428ccfa77fdbf P ea2086a88bfb40c7afe2181e79e15bf1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea2086a88bfb40c7afe2181e79e15bf1, State: Initialized, Role: FOLLOWER
I20260925 22:07:44.889118 19482 consensus_queue.cc:260] T ccdbabb1fb1f43c692d428ccfa77fdbf P ea2086a88bfb40c7afe2181e79e15bf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6de12817f7c94bdeb7e11241079e9778" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44715 } } peers { permanent_uuid: "dcb501810f564e5eb3dadc544cc81b81" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33761 } } peers { permanent_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45619 } }
I20260925 22:07:44.889415 19482 ts_tablet_manager.cc:1428] T ccdbabb1fb1f43c692d428ccfa77fdbf P ea2086a88bfb40c7afe2181e79e15bf1: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
Sep 25 22:07:44 dist-test-slave-7mqr krb5kdc[18822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374064, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:07:44.898588 19592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:44.881008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.65:43503)
0925 22:07:44.883277 (+ 2269us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:44.883312 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:44.884019 (+ 707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:44.884762 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:44.884767 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:44.885175 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:07:44.885475 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:44.885482 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.887374 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.887376 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:44.888871 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:44.888875 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.890860 (+ 1985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.890862 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:44.891027 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:44.891719 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:07:44.891729 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:07:44.893797 (+ 2068us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:07:44.896512 (+ 2715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:44.896519 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:44.896522 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:44.896860 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:44.897165 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:44.897166 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:44.897167 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:44.897330 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:44.897811 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:07:44.897818 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:07:44.898010 (+ 192us) client_negotiation.cc:769] Sending connection context
0925 22:07:44.898134 (+ 124us) client_negotiation.cc:240] Negotiation successful
0925 22:07:44.898326 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2228,"thread_start_us":105,"threads_started":1}
I20260925 22:07:44.898591 19593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:44.881122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:43503 (local address 127.5.0.126:34019)
0925 22:07:44.881358 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:44.881369 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:44.884204 (+ 2835us) server_negotiation.cc:407] Connection header received
0925 22:07:44.884352 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:44.884359 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:44.884431 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:44.884642 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:44.885669 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.887254 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.889015 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.890739 (+ 1724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.890885 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:44.893941 (+ 3056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:44.893947 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:44.893948 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:44.893963 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:44.896418 (+ 2455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:44.896967 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:44.896971 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:44.896971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:44.897088 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:44.897425 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:44.897426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:44.897426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:44.897729 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:44.897820 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:44.898217 (+ 397us) server_negotiation.cc:299] Negotiation successful
0925 22:07:44.898362 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":95,"threads_started":1}
I20260925 22:07:44.900501 19590 heartbeater.cc:347] Connected to a master server at 127.5.0.126:34019
I20260925 22:07:44.900605 19590 heartbeater.cc:464] Registering TS with master...
I20260925 22:07:44.900790 19590 heartbeater.cc:511] Master 127.5.0.126:34019 requested a full tablet report, sending...
I20260925 22:07:44.901703 19402 ts_manager.cc:194] Registered new tserver with Master: ea2086a88bfb40c7afe2181e79e15bf1 (127.5.0.65:45619)
I20260925 22:07:44.902882 19402 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.65@KRBTEST.COM'} at 127.5.0.65:43503
I20260925 22:07:44.903820 19590 heartbeater.cc:503] Master 127.5.0.126:34019 was elected leader, sending a full tablet report...
Sep 25 22:07:44 dist-test-slave-7mqr krb5kdc[18822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374064, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for kudu/127.5.0.65@KRBTEST.COM
I20260925 22:07:44.911818 19597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:44.902341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:45619 (local address 127.0.0.1:48828)
0925 22:07:44.902734 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:44.902768 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:44.902829 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:44.903389 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:44.903394 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:44.903715 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:07:44.903791 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:44.903793 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.905304 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.905305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:44.905929 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:44.905932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.906599 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.906602 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:44.906678 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:44.907238 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:07:44.907242 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:07:44.908638 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:07:44.910635 (+ 1997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:44.910641 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:44.910643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:44.910851 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:44.911072 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:44.911073 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:44.911073 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:44.911134 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:44.911486 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:07:44.911492 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:07:44.911569 (+ 77us) client_negotiation.cc:769] Sending connection context
0925 22:07:44.911648 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 22:07:44.911746 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":91,"threads_started":1}
I20260925 22:07:44.912073 19598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:44.902583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48828 (local address 127.5.0.65:45619)
0925 22:07:44.903024 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:44.903029 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:44.903038 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:07:44.903071 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:44.903073 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:44.903089 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:44.903330 (+ 241us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:44.903876 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.905213 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.906014 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:44.906488 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:44.906557 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:44.908721 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:44.908729 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:44.908729 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:44.908736 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:44.910537 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:44.910938 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:44.910941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:44.910942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:44.911006 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:44.911215 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:44.911216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:44.911216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:44.911393 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:44.911476 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:44.911703 (+ 227us) server_negotiation.cc:299] Negotiation successful
0925 22:07:44.911800 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":70,"threads_started":1}
I20260925 22:07:44.912418 19522 tablet_service.cc:1514] Processing DeleteTablet for tablet ccdbabb1fb1f43c692d428ccfa77fdbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:07:44 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:48828
I20260925 22:07:44.912948 19599 tablet_replica.cc:330] stopping tablet replica
I20260925 22:07:44.913132 19599 raft_consensus.cc:2229] T ccdbabb1fb1f43c692d428ccfa77fdbf P ea2086a88bfb40c7afe2181e79e15bf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:07:44.913205 19599 raft_consensus.cc:2258] T ccdbabb1fb1f43c692d428ccfa77fdbf P ea2086a88bfb40c7afe2181e79e15bf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:07:44.913509 19599 ts_tablet_manager.cc:1905] T ccdbabb1fb1f43c692d428ccfa77fdbf P ea2086a88bfb40c7afe2181e79e15bf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:07:44.915833 19599 ts_tablet_manager.cc:1918] T ccdbabb1fb1f43c692d428ccfa77fdbf P ea2086a88bfb40c7afe2181e79e15bf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:07:44.915931 19599 log.cc:1196] T ccdbabb1fb1f43c692d428ccfa77fdbf P ea2086a88bfb40c7afe2181e79e15bf1: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/wal/wals/ccdbabb1fb1f43c692d428ccfa77fdbf
I20260925 22:07:44.916154 19599 ts_tablet_manager.cc:1939] T ccdbabb1fb1f43c692d428ccfa77fdbf P ea2086a88bfb40c7afe2181e79e15bf1: Deleting consensus metadata
I20260925 22:07:44.916659 19388 catalog_manager.cc:4841] TS ea2086a88bfb40c7afe2181e79e15bf1 (127.5.0.65:45619): tablet ccdbabb1fb1f43c692d428ccfa77fdbf (table test-table [id=c0ca3c7804724cc5b67cc1881eb24658]) successfully deleted
W20260925 22:07:44.978890 19596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:07:44.979082 19596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:07:44.979104 19596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:07:44.980180 19596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:07:44.980212 19596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:07:44.980237 19596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.66
I20260925 22:07:44.981592 19596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:34219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.66:44715
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.0.66
--webserver_port=41497
--tserver_master_addrs=127.5.0.126:34019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.66
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:07:44.981775 19596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:07:44.981949 19596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:07:44.986819 19605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:44.986824 19607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:44.986847 19604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:07:44.987114 19596 server_base.cc:1034] running on GCE node
I20260925 22:07:44.987337 19596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:07:44.987684 19596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:07:44.988900 19596 hybrid_clock.cc:648] HybridClock initialized: now 1790374064988880 us; error 34 us; skew 500 ppm
Sep 25 22:07:44 dist-test-slave-7mqr krb5kdc[18822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374064, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:07:44.991505 19596 init.cc:377] Logged in from keytab as kudu/127.5.0.66@KRBTEST.COM (short username kudu)
I20260925 22:07:44.992057 19596 webserver.cc:460] Webserver started at http://127.5.0.66:41497/ using document root <none> and password file <none>
I20260925 22:07:44.992298 19596 fs_manager.cc:359] Metadata directory not provided
I20260925 22:07:44.992357 19596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:07:44.993746 19596 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:07:44.994364 19614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:44.994554 19596 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:07:44.994653 19596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/wal
uuid: "6de12817f7c94bdeb7e11241079e9778"
format_stamp: "Formatted at 2026-09-25 22:07:44 on dist-test-slave-7mqr"
server_key: "c1b19e914f4965384d2938c44f636007"
server_key_iv: "a4e7c1b74735afe4fa4291f3d9c0e614"
server_key_version: "encryptionkey@0"
I20260925 22:07:44.995045 19596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:07:45.004067 19596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:07:45.006325 19596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:07:45.006453 19596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:07:45.006628 19596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:07:45.006943 19596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:07:45.006978 19596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:45.007000 19596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:07:45.007054 19596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:45.012789 19596 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.66:44715
I20260925 22:07:45.012862 19727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.66:44715 every 8 connection(s)
I20260925 22:07:45.013216 19596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
I20260925 22:07:45.014711 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 19596
I20260925 22:07:45.014971 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.67:33761
--local_ip_for_outbound_sockets=127.5.0.67
--tserver_master_addrs=127.5.0.126:34019
--webserver_port=33617
--webserver_interface=127.5.0.67
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:34219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 22:07:45 dist-test-slave-7mqr krb5kdc[18822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374064, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:07:45.023989 19593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:45.013915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:53475 (local address 127.5.0.126:34019)
0925 22:07:45.013966 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:45.013969 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:45.014223 (+ 254us) server_negotiation.cc:407] Connection header received
0925 22:07:45.014750 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:45.014752 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:45.014764 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:45.014825 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:45.015947 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:45.016764 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:45.017776 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:45.018138 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:45.018216 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:45.020694 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:45.020696 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:45.020696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:45.020707 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:45.022499 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:45.022917 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:45.022919 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:45.022919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:45.022986 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:45.023240 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:45.023241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:45.023242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:45.023403 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:45.023477 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:45.023728 (+ 251us) server_negotiation.cc:299] Negotiation successful
0925 22:07:45.023826 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 22:07:45.024011 19730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:45.013895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.66:53475)
0925 22:07:45.014062 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:45.014127 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:45.014644 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:45.014937 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:45.014940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:45.015605 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:07:45.015817 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:45.015822 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:45.016885 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:45.016886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:45.017683 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:45.017685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:45.018228 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:45.018229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:45.018377 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:45.018908 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:07:45.018916 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:07:45.020591 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:07:45.022575 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:45.022580 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:45.022582 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:45.022843 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:45.023067 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:45.023068 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:45.023068 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:45.023173 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:45.023478 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:07:45.023482 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:07:45.023595 (+ 113us) client_negotiation.cc:769] Sending connection context
0925 22:07:45.023674 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 22:07:45.023791 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":73,"threads_started":1}
I20260925 22:07:45.024549 19728 heartbeater.cc:347] Connected to a master server at 127.5.0.126:34019
I20260925 22:07:45.024672 19728 heartbeater.cc:464] Registering TS with master...
I20260925 22:07:45.024865 19728 heartbeater.cc:511] Master 127.5.0.126:34019 requested a full tablet report, sending...
I20260925 22:07:45.025288 19402 ts_manager.cc:194] Registered new tserver with Master: 6de12817f7c94bdeb7e11241079e9778 (127.5.0.66:44715)
I20260925 22:07:45.025671 19402 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.66@KRBTEST.COM'} at 127.5.0.66:53475
W20260925 22:07:45.097230 19731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:07:45.097464 19731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:07:45.097489 19731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:07:45.098742 19731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:07:45.098784 19731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:07:45.098834 19731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.67
I20260925 22:07:45.100320 19731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:34219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.67:33761
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.0.67
--webserver_port=33617
--tserver_master_addrs=127.5.0.126:34019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.67
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:07:45.100553 19731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:07:45.100778 19731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:07:45.105481 19739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:45.105485 19737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:45.105512 19736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:07:45.105831 19731 server_base.cc:1034] running on GCE node
I20260925 22:07:45.106005 19731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:07:45.106238 19731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:07:45.107398 19731 hybrid_clock.cc:648] HybridClock initialized: now 1790374065107376 us; error 32 us; skew 500 ppm
Sep 25 22:07:45 dist-test-slave-7mqr krb5kdc[18822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374065, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:07:45.109858 19731 init.cc:377] Logged in from keytab as kudu/127.5.0.67@KRBTEST.COM (short username kudu)
I20260925 22:07:45.110277 19731 webserver.cc:460] Webserver started at http://127.5.0.67:33617/ using document root <none> and password file <none>
I20260925 22:07:45.110455 19731 fs_manager.cc:359] Metadata directory not provided
I20260925 22:07:45.110507 19731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:07:45.111759 19731 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:07:45.112361 19746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:45.112527 19731 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:45.112612 19731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/wal
uuid: "dcb501810f564e5eb3dadc544cc81b81"
format_stamp: "Formatted at 2026-09-25 22:07:44 on dist-test-slave-7mqr"
server_key: "4c4f95fcc0c5436faecefcf7ff29589b"
server_key_iv: "34ccab7f0deca77aa655067ee11d3747"
server_key_version: "encryptionkey@0"
I20260925 22:07:45.112943 19731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:07:45.127538 19731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:07:45.129930 19731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:07:45.130101 19731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:07:45.130335 19731 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:07:45.130641 19731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:07:45.130695 19731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:45.130738 19731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:07:45.130764 19731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:45.137503 19731 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.67:33761
I20260925 22:07:45.137544 19859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.67:33761 every 8 connection(s)
I20260925 22:07:45.137885 19731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
I20260925 22:07:45.140237 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 19731
Sep 25 22:07:45 dist-test-slave-7mqr krb5kdc[18822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374065, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:07:45.148710 19593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:45.138701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:56027 (local address 127.5.0.126:34019)
0925 22:07:45.138940 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:45.138942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:45.138958 (+ 16us) server_negotiation.cc:407] Connection header received
0925 22:07:45.139472 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:45.139474 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:45.139487 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:45.139563 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:45.140814 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:45.141312 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:45.142333 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:45.142542 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:45.142635 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:45.145394 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:45.145396 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:45.145397 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:45.145405 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:45.147117 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:45.147639 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:45.147641 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:45.147641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:45.147702 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:45.147988 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:45.147989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:45.147990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:45.148151 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:45.148228 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:45.148542 (+ 314us) server_negotiation.cc:299] Negotiation successful
0925 22:07:45.148638 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205}
I20260925 22:07:45.149293 19862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:45.138689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.67:56027)
0925 22:07:45.138864 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:45.138900 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:45.139367 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:45.139624 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:45.139628 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:45.140285 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:07:45.140610 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:45.140734 (+ 124us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:45.141417 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:45.141418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:45.142229 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:45.142231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:45.142638 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:45.142639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:45.142766 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:45.143350 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:07:45.143358 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:07:45.145292 (+ 1934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:07:45.147211 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:45.147218 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:45.147221 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:45.147529 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:45.147777 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:45.147778 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:45.147778 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:45.147933 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:45.148230 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:07:45.148236 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:07:45.148374 (+ 138us) client_negotiation.cc:769] Sending connection context
0925 22:07:45.148868 (+ 494us) client_negotiation.cc:240] Negotiation successful
0925 22:07:45.149045 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":89,"threads_started":1}
I20260925 22:07:45.150945 19860 heartbeater.cc:347] Connected to a master server at 127.5.0.126:34019
I20260925 22:07:45.151064 19860 heartbeater.cc:464] Registering TS with master...
I20260925 22:07:45.151244 19860 heartbeater.cc:511] Master 127.5.0.126:34019 requested a full tablet report, sending...
I20260925 22:07:45.151655 19402 ts_manager.cc:194] Registered new tserver with Master: dcb501810f564e5eb3dadc544cc81b81 (127.5.0.67:33761)
I20260925 22:07:45.152247 19402 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.67@KRBTEST.COM'} at 127.5.0.67:56027
I20260925 22:07:45.152698 19864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:45.140657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38484 (local address 127.5.0.126:34019)
0925 22:07:45.143369 (+ 2712us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:45.143371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:45.143383 (+ 12us) server_negotiation.cc:407] Connection header received
0925 22:07:45.143407 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:45.143408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:45.143423 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:45.143508 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:45.144296 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:45.144770 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:45.145769 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:45.146161 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:45.146237 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:45.147965 (+ 1728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:45.147966 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:45.147966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:45.147974 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:45.151340 (+ 3366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:45.151811 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:45.151812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:45.151813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:45.151870 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:45.152107 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:45.152108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:45.152108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:45.152222 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:45.152455 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:45.152498 (+ 43us) server_negotiation.cc:299] Negotiation successful
0925 22:07:45.152575 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2689,"thread_start_us":55,"threads_started":1}
I20260925 22:07:45.153680 5121 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:07:45.156912 19864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:45.154036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38498 (local address 127.5.0.126:34019)
0925 22:07:45.154076 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:45.154078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:45.154175 (+ 97us) server_negotiation.cc:407] Connection header received
0925 22:07:45.154253 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:45.154254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:45.154279 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:45.154333 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:07:45.154480 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:45.154930 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:45.156220 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:45.156399 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:45.156452 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:45.156647 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:07:45.156731 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:07:45.156788 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:45.156852 (+ 64us) server_negotiation.cc:299] Negotiation successful
0925 22:07:45.156869 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 22:07:45.157162 19402 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:38498:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:07:45.158849 19402 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:07:45.161243 19522 tablet_service.cc:1467] Processing CreateTablet for tablet b2c88155a71a4eef8649df0ffaa9d356 (DEFAULT_TABLE table=test-table [id=7fbf2b737e7c4d9788c46a852c196b06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:07:45.161434 19522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2c88155a71a4eef8649df0ffaa9d356. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:07:45.162851 19482 tablet_bootstrap.cc:492] T b2c88155a71a4eef8649df0ffaa9d356 P ea2086a88bfb40c7afe2181e79e15bf1: Bootstrap starting.
I20260925 22:07:45.163163 19482 tablet_bootstrap.cc:654] T b2c88155a71a4eef8649df0ffaa9d356 P ea2086a88bfb40c7afe2181e79e15bf1: Neither blocks nor log segments found. Creating new log.
I20260925 22:07:45.163659 19482 tablet_bootstrap.cc:492] T b2c88155a71a4eef8649df0ffaa9d356 P ea2086a88bfb40c7afe2181e79e15bf1: No bootstrap required, opened a new log
I20260925 22:07:45.163704 19482 ts_tablet_manager.cc:1397] T b2c88155a71a4eef8649df0ffaa9d356 P ea2086a88bfb40c7afe2181e79e15bf1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 22:07:45.164072 19482 raft_consensus.cc:348] T b2c88155a71a4eef8649df0ffaa9d356 P ea2086a88bfb40c7afe2181e79e15bf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcb501810f564e5eb3dadc544cc81b81" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33761 } } peers { permanent_uuid: "6de12817f7c94bdeb7e11241079e9778" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44715 } } peers { permanent_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45619 } }
I20260925 22:07:45.164136 19482 raft_consensus.cc:374] T b2c88155a71a4eef8649df0ffaa9d356 P ea2086a88bfb40c7afe2181e79e15bf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:07:45.164192 19482 raft_consensus.cc:729] T b2c88155a71a4eef8649df0ffaa9d356 P ea2086a88bfb40c7afe2181e79e15bf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea2086a88bfb40c7afe2181e79e15bf1, State: Initialized, Role: FOLLOWER
I20260925 22:07:45.164239 19482 consensus_queue.cc:260] T b2c88155a71a4eef8649df0ffaa9d356 P ea2086a88bfb40c7afe2181e79e15bf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcb501810f564e5eb3dadc544cc81b81" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33761 } } peers { permanent_uuid: "6de12817f7c94bdeb7e11241079e9778" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44715 } } peers { permanent_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45619 } }
I20260925 22:07:45.164345 19482 ts_tablet_manager.cc:1428] T b2c88155a71a4eef8649df0ffaa9d356 P ea2086a88bfb40c7afe2181e79e15bf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 22:07:45.165331 19597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:45.160719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:33761 (local address 127.0.0.1:56550)
0925 22:07:45.160787 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:45.160799 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:45.160838 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:45.161324 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:45.161325 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:45.161327 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:07:45.161373 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:45.161375 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:45.162645 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:45.162646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:45.164104 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:45.164107 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:45.165105 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:45.165106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:45.165176 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:45.165198 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 22:07:45.165227 (+ 29us) client_negotiation.cc:240] Negotiation successful
0925 22:07:45.165265 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260925 22:07:45.165658 19867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:45.160918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56550 (local address 127.5.0.67:33761)
0925 22:07:45.161095 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:45.161100 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:45.161109 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:07:45.161179 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:45.161182 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:45.161199 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:45.161314 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:07:45.161433 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:45.162537 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:45.164198 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:45.165011 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:45.165427 (+ 416us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:45.165498 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:45.165529 (+ 31us) server_negotiation.cc:299] Negotiation successful
0925 22:07:45.165574 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":74,"threads_started":1}
I20260925 22:07:45.166807 19866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:45.160734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:44715 (local address 127.0.0.1:48264)
0925 22:07:45.160912 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:45.160922 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:45.161199 (+ 277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:45.161455 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:45.161456 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:45.161459 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:07:45.161501 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:45.161502 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:45.163372 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:45.163373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:45.165466 (+ 2093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:45.165468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:45.166595 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:45.166596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:45.166657 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:45.166660 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:07:45.166710 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 22:07:45.166745 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":110,"threads_started":1}
I20260925 22:07:45.166908 19868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:45.161040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48264 (local address 127.5.0.66:44715)
0925 22:07:45.161237 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:45.161241 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:45.161250 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:07:45.161287 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:45.161289 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:45.161304 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:45.161374 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:07:45.161578 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:45.162739 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:45.165607 (+ 2868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:45.166265 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:45.166305 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:45.166337 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:45.166731 (+ 394us) server_negotiation.cc:299] Negotiation successful
0925 22:07:45.166790 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":105,"threads_started":1}
I20260925 22:07:45.167014 19794 tablet_service.cc:1467] Processing CreateTablet for tablet b2c88155a71a4eef8649df0ffaa9d356 (DEFAULT_TABLE table=test-table [id=7fbf2b737e7c4d9788c46a852c196b06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:07:45.167272 19794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2c88155a71a4eef8649df0ffaa9d356. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:07:45.167567 19662 tablet_service.cc:1467] Processing CreateTablet for tablet b2c88155a71a4eef8649df0ffaa9d356 (DEFAULT_TABLE table=test-table [id=7fbf2b737e7c4d9788c46a852c196b06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:07:45.167879 19662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2c88155a71a4eef8649df0ffaa9d356. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:07:45.169970 19870 tablet_bootstrap.cc:492] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81: Bootstrap starting.
I20260925 22:07:45.170480 19870 tablet_bootstrap.cc:654] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81: Neither blocks nor log segments found. Creating new log.
I20260925 22:07:45.170908 19870 log.cc:824] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81: Log is configured to *not* fsync() on all Append() calls
I20260925 22:07:45.171546 19870 tablet_bootstrap.cc:492] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81: No bootstrap required, opened a new log
I20260925 22:07:45.171591 19870 ts_tablet_manager.cc:1397] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:07:45.171931 19872 tablet_bootstrap.cc:492] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778: Bootstrap starting.
I20260925 22:07:45.172587 19872 tablet_bootstrap.cc:654] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778: Neither blocks nor log segments found. Creating new log.
I20260925 22:07:45.173138 19872 log.cc:824] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778: Log is configured to *not* fsync() on all Append() calls
I20260925 22:07:45.173768 19870 raft_consensus.cc:348] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcb501810f564e5eb3dadc544cc81b81" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33761 } } peers { permanent_uuid: "6de12817f7c94bdeb7e11241079e9778" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44715 } } peers { permanent_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45619 } }
I20260925 22:07:45.173949 19870 raft_consensus.cc:374] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:07:45.173974 19872 tablet_bootstrap.cc:492] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778: No bootstrap required, opened a new log
I20260925 22:07:45.173988 19870 raft_consensus.cc:729] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcb501810f564e5eb3dadc544cc81b81, State: Initialized, Role: FOLLOWER
I20260925 22:07:45.174039 19872 ts_tablet_manager.cc:1397] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:07:45.174139 19870 consensus_queue.cc:260] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcb501810f564e5eb3dadc544cc81b81" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33761 } } peers { permanent_uuid: "6de12817f7c94bdeb7e11241079e9778" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44715 } } peers { permanent_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45619 } }
I20260925 22:07:45.174418 19870 ts_tablet_manager.cc:1428] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 22:07:45.174459 19860 heartbeater.cc:503] Master 127.5.0.126:34019 was elected leader, sending a full tablet report...
I20260925 22:07:45.175834 19872 raft_consensus.cc:348] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcb501810f564e5eb3dadc544cc81b81" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33761 } } peers { permanent_uuid: "6de12817f7c94bdeb7e11241079e9778" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44715 } } peers { permanent_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45619 } }
I20260925 22:07:45.176034 19872 raft_consensus.cc:374] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:07:45.176062 19872 raft_consensus.cc:729] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6de12817f7c94bdeb7e11241079e9778, State: Initialized, Role: FOLLOWER
I20260925 22:07:45.176191 19872 consensus_queue.cc:260] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcb501810f564e5eb3dadc544cc81b81" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33761 } } peers { permanent_uuid: "6de12817f7c94bdeb7e11241079e9778" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44715 } } peers { permanent_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45619 } }
I20260925 22:07:45.176460 19872 ts_tablet_manager.cc:1428] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 22:07:45.176493 19728 heartbeater.cc:503] Master 127.5.0.126:34019 was elected leader, sending a full tablet report...
I20260925 22:07:45.189917 19875 raft_consensus.cc:482] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:07:45.190016 19875 raft_consensus.cc:504] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcb501810f564e5eb3dadc544cc81b81" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33761 } } peers { permanent_uuid: "6de12817f7c94bdeb7e11241079e9778" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44715 } } peers { permanent_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45619 } }
I20260925 22:07:45.190426 19875 leader_election.cc:288] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dcb501810f564e5eb3dadc544cc81b81 (127.5.0.67:33761), ea2086a88bfb40c7afe2181e79e15bf1 (127.5.0.65:45619)
I20260925 22:07:45.194025 19867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:45.190649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:45187 (local address 127.5.0.67:33761)
0925 22:07:45.190809 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:45.190811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:45.190927 (+ 116us) server_negotiation.cc:407] Connection header received
0925 22:07:45.191025 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:45.191027 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:45.191038 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:45.191086 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:07:45.191285 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:45.192145 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:45.193286 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:45.193663 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:45.193756 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:45.193766 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:45.193922 (+ 156us) server_negotiation.cc:299] Negotiation successful
0925 22:07:45.193962 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260925 22:07:45.194036 19876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:45.190592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:33761 (local address 127.5.0.66:45187)
0925 22:07:45.190876 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:45.190886 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:45.190947 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:45.191139 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:45.191140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:45.191141 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:07:45.191202 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:45.191204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:45.192235 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:45.192236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:45.193204 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:45.193206 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:45.193759 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:45.193760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:45.193835 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:45.193839 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:07:45.193913 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 22:07:45.193955 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":61,"thread_start_us":112,"threads_started":1}
I20260925 22:07:45.194290 19730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:45.190590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:45619 (local address 127.5.0.66:36779)
0925 22:07:45.190674 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:45.190687 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:45.190727 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:45.190905 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:45.190907 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:45.190909 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:07:45.190991 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:45.190993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:45.191750 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:45.191751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:45.192855 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:45.192858 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:45.193946 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:45.193947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:45.194070 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:45.194075 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:07:45.194136 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 22:07:45.194190 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260925 22:07:45.194325 19814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2c88155a71a4eef8649df0ffaa9d356" candidate_uuid: "6de12817f7c94bdeb7e11241079e9778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcb501810f564e5eb3dadc544cc81b81" is_pre_election: true
I20260925 22:07:45.194491 19814 raft_consensus.cc:2454] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6de12817f7c94bdeb7e11241079e9778 in term 0.
I20260925 22:07:45.194736 19615 leader_election.cc:302] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6de12817f7c94bdeb7e11241079e9778, dcb501810f564e5eb3dadc544cc81b81; no voters:
I20260925 22:07:45.194865 19875 raft_consensus.cc:2781] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:07:45.194945 19875 raft_consensus.cc:482] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:07:45.194981 19875 raft_consensus.cc:3037] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:07:45.195062 19598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:45.190700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:36779 (local address 127.5.0.65:45619)
0925 22:07:45.190742 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:45.190744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:45.190753 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:07:45.190781 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:45.190782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:45.190809 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:45.190878 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:07:45.191080 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:45.191661 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:45.192954 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:45.193740 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:45.193820 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:45.193909 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:45.194897 (+ 988us) server_negotiation.cc:299] Negotiation successful
0925 22:07:45.194958 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 22:07:45.195535 19532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2c88155a71a4eef8649df0ffaa9d356" candidate_uuid: "6de12817f7c94bdeb7e11241079e9778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" is_pre_election: true
I20260925 22:07:45.195690 19532 raft_consensus.cc:2454] T b2c88155a71a4eef8649df0ffaa9d356 P ea2086a88bfb40c7afe2181e79e15bf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6de12817f7c94bdeb7e11241079e9778 in term 0.
I20260925 22:07:45.196077 19875 raft_consensus.cc:504] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcb501810f564e5eb3dadc544cc81b81" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33761 } } peers { permanent_uuid: "6de12817f7c94bdeb7e11241079e9778" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44715 } } peers { permanent_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45619 } }
I20260925 22:07:45.196224 19875 leader_election.cc:288] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [CANDIDATE]: Term 1 election: Requested vote from peers dcb501810f564e5eb3dadc544cc81b81 (127.5.0.67:33761), ea2086a88bfb40c7afe2181e79e15bf1 (127.5.0.65:45619)
I20260925 22:07:45.196353 19814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2c88155a71a4eef8649df0ffaa9d356" candidate_uuid: "6de12817f7c94bdeb7e11241079e9778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcb501810f564e5eb3dadc544cc81b81"
I20260925 22:07:45.196379 19532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2c88155a71a4eef8649df0ffaa9d356" candidate_uuid: "6de12817f7c94bdeb7e11241079e9778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea2086a88bfb40c7afe2181e79e15bf1"
I20260925 22:07:45.196452 19814 raft_consensus.cc:3037] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:07:45.196478 19532 raft_consensus.cc:3037] T b2c88155a71a4eef8649df0ffaa9d356 P ea2086a88bfb40c7afe2181e79e15bf1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:07:45.197144 19814 raft_consensus.cc:2454] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6de12817f7c94bdeb7e11241079e9778 in term 1.
I20260925 22:07:45.197297 19615 leader_election.cc:302] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6de12817f7c94bdeb7e11241079e9778, dcb501810f564e5eb3dadc544cc81b81; no voters:
I20260925 22:07:45.197376 19875 raft_consensus.cc:2781] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:07:45.197556 19875 raft_consensus.cc:686] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [term 1 LEADER]: Becoming Leader. State: Replica: 6de12817f7c94bdeb7e11241079e9778, State: Running, Role: LEADER
I20260925 22:07:45.197551 19532 raft_consensus.cc:2454] T b2c88155a71a4eef8649df0ffaa9d356 P ea2086a88bfb40c7afe2181e79e15bf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6de12817f7c94bdeb7e11241079e9778 in term 1.
I20260925 22:07:45.197685 19875 consensus_queue.cc:237] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcb501810f564e5eb3dadc544cc81b81" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33761 } } peers { permanent_uuid: "6de12817f7c94bdeb7e11241079e9778" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44715 } } peers { permanent_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45619 } }
I20260925 22:07:45.198740 19402 catalog_manager.cc:5495] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6de12817f7c94bdeb7e11241079e9778 (127.5.0.66). New cstate: current_term: 1 leader_uuid: "6de12817f7c94bdeb7e11241079e9778" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcb501810f564e5eb3dadc544cc81b81" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6de12817f7c94bdeb7e11241079e9778" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45619 } health_report { overall_health: UNKNOWN } } }
I20260925 22:07:45.208217 19868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:45.205324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48280 (local address 127.5.0.66:44715)
0925 22:07:45.205392 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:45.205394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:45.205404 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:07:45.205442 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:45.205443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:45.205465 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:45.205548 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:07:45.205672 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:45.206537 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:45.207262 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:45.207621 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:45.207693 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:45.207795 (+ 102us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:07:45.207955 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:07:45.208049 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:45.208066 (+ 17us) server_negotiation.cc:299] Negotiation successful
0925 22:07:45.208093 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 22:07:45.209960 19814 raft_consensus.cc:1261] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81 [term 1 FOLLOWER]: Refusing update from remote peer 6de12817f7c94bdeb7e11241079e9778: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:07:45.209993 19532 raft_consensus.cc:1261] T b2c88155a71a4eef8649df0ffaa9d356 P ea2086a88bfb40c7afe2181e79e15bf1 [term 1 FOLLOWER]: Refusing update from remote peer 6de12817f7c94bdeb7e11241079e9778: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:07:45.210345 19877 consensus_queue.cc:1035] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dcb501810f564e5eb3dadc544cc81b81" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:07:45.210508 19875 consensus_queue.cc:1035] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:07:45.213243 19881 mvcc.cc:205] Tried to move back new op lower bound from 7333372171097206784 to 7333372171050614784. Current Snapshot: MvccSnapshot[applied={T|T < 7333372171097206784}]
I20260925 22:07:45.213364 19882 mvcc.cc:205] Tried to move back new op lower bound from 7333372171097206784 to 7333372171050614784. Current Snapshot: MvccSnapshot[applied={T|T < 7333372171097206784}]
I20260925 22:07:45.215051 19400 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:38498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:07:45.215150 19400 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:38498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:07:45.215976 19400 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572: Sending DeleteTablet for 3 replicas of tablet b2c88155a71a4eef8649df0ffaa9d356
I20260925 22:07:45.216255 19662 tablet_service.cc:1514] Processing DeleteTablet for tablet b2c88155a71a4eef8649df0ffaa9d356 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:07:45 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:48264
I20260925 22:07:45.216284 19522 tablet_service.cc:1514] Processing DeleteTablet for tablet b2c88155a71a4eef8649df0ffaa9d356 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:07:45 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:48828
I20260925 22:07:45.216388 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 19440
I20260925 22:07:45.216408 19599 tablet_replica.cc:330] T b2c88155a71a4eef8649df0ffaa9d356 P ea2086a88bfb40c7afe2181e79e15bf1: stopping tablet replica
I20260925 22:07:45.216460 19599 raft_consensus.cc:2229] T b2c88155a71a4eef8649df0ffaa9d356 P ea2086a88bfb40c7afe2181e79e15bf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:07:45.216444 19794 tablet_service.cc:1514] Processing DeleteTablet for tablet b2c88155a71a4eef8649df0ffaa9d356 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:07:45 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:56550
I20260925 22:07:45.216511 19599 raft_consensus.cc:2258] T b2c88155a71a4eef8649df0ffaa9d356 P ea2086a88bfb40c7afe2181e79e15bf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:07:45.217248 19890 tablet_replica.cc:330] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81: stopping tablet replica
I20260925 22:07:45.217350 19890 raft_consensus.cc:2229] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:07:45.217447 19890 raft_consensus.cc:2258] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:07:45.217829 19890 ts_tablet_manager.cc:1905] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:07:45.219424 19890 ts_tablet_manager.cc:1918] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:07:45.219511 19890 log.cc:1196] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/wal/wals/b2c88155a71a4eef8649df0ffaa9d356
I20260925 22:07:45.219798 19890 ts_tablet_manager.cc:1939] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81: Deleting consensus metadata
I20260925 22:07:45.220211 19386 catalog_manager.cc:4841] TS dcb501810f564e5eb3dadc544cc81b81 (127.5.0.67:33761): tablet b2c88155a71a4eef8649df0ffaa9d356 (table test-table [id=7fbf2b737e7c4d9788c46a852c196b06]) successfully deleted
W20260925 22:07:45.223702 19388 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.0.65:45619 (error 108)
I20260925 22:07:45.224104 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 19596
I20260925 22:07:45.224251 19866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:45.224099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:45619 (local address 127.0.0.1:48832)
0925 22:07:45.224154 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:45.224214 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.65:45619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260925 22:07:45.225123 19388 catalog_manager.cc:4584] TS ea2086a88bfb40c7afe2181e79e15bf1 (127.5.0.65:45619): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b2c88155a71a4eef8649df0ffaa9d356: Network error: Client connection negotiation failed: client connection to 127.5.0.65:45619: connect: Connection refused (error 111)
I20260925 22:07:45.228574 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 19731
I20260925 22:07:45.230315 19866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:45.228607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:44715 (local address 127.0.0.1:48294)
0925 22:07:45.230255 (+ 1648us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:45.230287 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.66:44715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1627}
I20260925 22:07:45.232879 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 19369
I20260925 22:07:45.238130 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:34019
--webserver_interface=127.5.0.126
--webserver_port=44265
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:34219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:34019
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:07:45.318207 19892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:07:45.318421 19892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:07:45.318444 19892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:07:45.319623 19892 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:07:45.319666 19892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:07:45.319680 19892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:07:45.319693 19892 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:07:45.319747 19892 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:07:45.321202 19892 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:34219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:34019
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:34019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=44265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:07:45.321461 19892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:07:45.321697 19892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:07:45.326475 19900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:45.326504 19898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:45.326534 19897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:07:45.326696 19892 server_base.cc:1034] running on GCE node
I20260925 22:07:45.326915 19892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:07:45.327162 19892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:07:45.328296 19892 hybrid_clock.cc:648] HybridClock initialized: now 1790374065328277 us; error 30 us; skew 500 ppm
Sep 25 22:07:45 dist-test-slave-7mqr krb5kdc[18822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374065, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:07:45.330797 19892 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:07:45.331259 19892 webserver.cc:460] Webserver started at http://127.5.0.126:44265/ using document root <none> and password file <none>
I20260925 22:07:45.331489 19892 fs_manager.cc:359] Metadata directory not provided
I20260925 22:07:45.331535 19892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:07:45.332933 19892 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:07:45.333592 19907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:45.333760 19892 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:07:45.333864 19892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "475894d649af404baa1eb2718c7b6572"
format_stamp: "Formatted at 2026-09-25 22:07:43 on dist-test-slave-7mqr"
server_key: "630a3b972c59128422a37a0c489e9282"
server_key_iv: "36ca75de1ef56b8114a0b21502ee1d11"
server_key_version: "encryptionkey@0"
I20260925 22:07:45.334204 19892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:07:45.356396 19892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:07:45.358819 19892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:07:45.358991 19892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:07:45.363315 19892 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:34019
I20260925 22:07:45.363353 19959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:34019 every 8 connection(s)
I20260925 22:07:45.363760 19892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:07:45.365338 19960 sys_catalog.cc:263] Verifying existing consensus state
I20260925 22:07:45.365927 19960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572: Bootstrap starting.
I20260925 22:07:45.373024 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 19892
I20260925 22:07:45.373373 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.65:45619
--local_ip_for_outbound_sockets=127.5.0.65
--tserver_master_addrs=127.5.0.126:34019
--webserver_port=39111
--webserver_interface=127.5.0.65
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:34219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:07:45.374591 19960 log.cc:824] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572: Log is configured to *not* fsync() on all Append() calls
I20260925 22:07:45.377154 19960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 22:07:45.377465 19960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572: Bootstrap complete.
I20260925 22:07:45.379786 19960 raft_consensus.cc:348] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "475894d649af404baa1eb2718c7b6572" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34019 } }
I20260925 22:07:45.380084 19960 raft_consensus.cc:729] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 475894d649af404baa1eb2718c7b6572, State: Initialized, Role: FOLLOWER
I20260925 22:07:45.380256 19960 consensus_queue.cc:260] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "475894d649af404baa1eb2718c7b6572" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34019 } }
I20260925 22:07:45.380316 19960 raft_consensus.cc:388] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:07:45.380373 19960 raft_consensus.cc:482] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:07:45.380422 19960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [term 2 FOLLOWER]: Advancing to term 3
I20260925 22:07:45.381430 19960 raft_consensus.cc:504] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "475894d649af404baa1eb2718c7b6572" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34019 } }
I20260925 22:07:45.381516 19960 leader_election.cc:302] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 475894d649af404baa1eb2718c7b6572; no voters:
I20260925 22:07:45.381733 19960 leader_election.cc:288] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260925 22:07:45.381768 19964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [term 3 FOLLOWER]: Leader election won for term 3
I20260925 22:07:45.381875 19964 raft_consensus.cc:686] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [term 3 LEADER]: Becoming Leader. State: Replica: 475894d649af404baa1eb2718c7b6572, State: Running, Role: LEADER
I20260925 22:07:45.381978 19964 consensus_queue.cc:237] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "475894d649af404baa1eb2718c7b6572" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34019 } }
I20260925 22:07:45.382009 19960 sys_catalog.cc:564] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:07:45.382256 19965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "475894d649af404baa1eb2718c7b6572" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "475894d649af404baa1eb2718c7b6572" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34019 } } }
I20260925 22:07:45.382525 19965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [sys.catalog]: This master's current role is: LEADER
I20260925 22:07:45.382587 19966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 475894d649af404baa1eb2718c7b6572. Latest consensus state: current_term: 3 leader_uuid: "475894d649af404baa1eb2718c7b6572" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "475894d649af404baa1eb2718c7b6572" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34019 } } }
I20260925 22:07:45.382676 19966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 475894d649af404baa1eb2718c7b6572 [sys.catalog]: This master's current role is: LEADER
I20260925 22:07:45.382880 19974 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:07:45.383785 19974 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:07:45.383966 19974 catalog_manager.cc:1259] Loaded cluster ID: 3fc71eb117d04324bef4c9c870cd2682
I20260925 22:07:45.384156 19974 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260925 22:07:45.384416 19974 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fc507f7c9ad google::LogMessage::Fail() at ??:0
@ 0x7fc507f7f88a google::LogMessage::SendToLog() at ??:0
@ 0x7fc507f7c4de google::LogMessage::Flush() at ??:0
@ 0x7fc507f7da19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fc50fea10f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fc50feaa8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fc509e4a0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fc509e3f954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc50aca36db start_thread at ??:0
@ 0x7fc506a7a61f clone at ??:0
W20260925 22:07:45.490120 19962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:07:45.490386 19962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:07:45.490422 19962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:07:45.492157 19962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:07:45.492218 19962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:07:45.492260 19962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.65
I20260925 22:07:45.494462 19962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:34219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.65:45619
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.0.65
--webserver_port=39111
--tserver_master_addrs=127.5.0.126:34019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.65
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:07:45.494741 19962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:07:45.495025 19962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:07:45.500465 19999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:45.500510 19997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:45.500625 19996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:07:45.500662 19962 server_base.cc:1034] running on GCE node
I20260925 22:07:45.500877 19962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:07:45.501132 19962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:07:45.502301 19962 hybrid_clock.cc:648] HybridClock initialized: now 1790374065502278 us; error 38 us; skew 500 ppm
Sep 25 22:07:45 dist-test-slave-7mqr krb5kdc[18822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374065, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:07:45.504833 19962 init.cc:377] Logged in from keytab as kudu/127.5.0.65@KRBTEST.COM (short username kudu)
I20260925 22:07:45.505214 19962 webserver.cc:460] Webserver started at http://127.5.0.65:39111/ using document root <none> and password file <none>
I20260925 22:07:45.505466 19962 fs_manager.cc:359] Metadata directory not provided
I20260925 22:07:45.505715 19962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:07:45.507362 19962 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 22:07:45.508132 20006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:45.508319 19962 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:07:45.508423 19962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/wal
uuid: "ea2086a88bfb40c7afe2181e79e15bf1"
format_stamp: "Formatted at 2026-09-25 22:07:43 on dist-test-slave-7mqr"
server_key: "0e2dd152a80271c53feb36a56c2bb1b7"
server_key_iv: "fbad3187a14e3f4cbbb12307ea495257"
server_key_version: "encryptionkey@0"
I20260925 22:07:45.508791 19962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:07:45.522864 19962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:07:45.525326 19962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:07:45.525492 19962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:07:45.525738 19962 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:07:45.526182 20013 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 22:07:45.528247 19962 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 22:07:45.528306 19962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260925 22:07:45.528362 19962 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 22:07:45.528998 19962 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 22:07:45.529053 19962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260925 22:07:45.529124 20013 tablet_bootstrap.cc:492] T b2c88155a71a4eef8649df0ffaa9d356 P ea2086a88bfb40c7afe2181e79e15bf1: Bootstrap starting.
I20260925 22:07:45.536183 19962 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.65:45619
I20260925 22:07:45.536338 20120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.65:45619 every 8 connection(s)
I20260925 22:07:45.536612 19962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
I20260925 22:07:45.539846 20013 log.cc:824] T b2c88155a71a4eef8649df0ffaa9d356 P ea2086a88bfb40c7afe2181e79e15bf1: Log is configured to *not* fsync() on all Append() calls
I20260925 22:07:45.540167 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 19962
I20260925 22:07:45.540464 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.66:44715
--local_ip_for_outbound_sockets=127.5.0.66
--tserver_master_addrs=127.5.0.126:34019
--webserver_port=41497
--webserver_interface=127.5.0.66
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:34219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:07:45.542114 20013 tablet_bootstrap.cc:492] T b2c88155a71a4eef8649df0ffaa9d356 P ea2086a88bfb40c7afe2181e79e15bf1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 22:07:45.542395 20013 tablet_bootstrap.cc:492] T b2c88155a71a4eef8649df0ffaa9d356 P ea2086a88bfb40c7afe2181e79e15bf1: Bootstrap complete.
I20260925 22:07:45.543566 20013 ts_tablet_manager.cc:1397] T b2c88155a71a4eef8649df0ffaa9d356 P ea2086a88bfb40c7afe2181e79e15bf1: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.007s
I20260925 22:07:45.545074 20013 raft_consensus.cc:348] T b2c88155a71a4eef8649df0ffaa9d356 P ea2086a88bfb40c7afe2181e79e15bf1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcb501810f564e5eb3dadc544cc81b81" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33761 } } peers { permanent_uuid: "6de12817f7c94bdeb7e11241079e9778" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44715 } } peers { permanent_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45619 } }
I20260925 22:07:45.545392 20013 raft_consensus.cc:729] T b2c88155a71a4eef8649df0ffaa9d356 P ea2086a88bfb40c7afe2181e79e15bf1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea2086a88bfb40c7afe2181e79e15bf1, State: Initialized, Role: FOLLOWER
I20260925 22:07:45.545560 20013 consensus_queue.cc:260] T b2c88155a71a4eef8649df0ffaa9d356 P ea2086a88bfb40c7afe2181e79e15bf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcb501810f564e5eb3dadc544cc81b81" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33761 } } peers { permanent_uuid: "6de12817f7c94bdeb7e11241079e9778" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44715 } } peers { permanent_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45619 } }
I20260925 22:07:45.545840 20013 ts_tablet_manager.cc:1428] T b2c88155a71a4eef8649df0ffaa9d356 P ea2086a88bfb40c7afe2181e79e15bf1: Time spent starting tablet: real 0.002s user 0.004s sys 0.001s
W20260925 22:07:45.587615 20123 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:07:45.537330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.65:44799)
0925 22:07:45.537502 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:45.537547 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:45.537996 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:45.587393 (+ 49397us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":93,"threads_started":1}
W20260925 22:07:45.587833 20121 heartbeater.cc:644] Failed to heartbeat to 127.5.0.126:34019 (0 consecutive failures): Network error: Failed to ping master at 127.5.0.126:34019: Client connection negotiation failed: client connection to 127.5.0.126:34019: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 22:07:45.588207 20123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:45.588075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.65:34463)
0925 22:07:45.588140 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:45.588181 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 22:07:45.588580 20123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:45.588500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.65:50997)
0925 22:07:45.588544 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:45.588566 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260925 22:07:45.588711 20121 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 22:07:45.623019 20125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:07:45.623236 20125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:07:45.623260 20125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:07:45.624426 20125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:07:45.624470 20125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:07:45.624496 20125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.66
I20260925 22:07:45.625967 20125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:34219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.66:44715
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.0.66
--webserver_port=41497
--tserver_master_addrs=127.5.0.126:34019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.66
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:07:45.626157 20125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:07:45.626332 20125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:07:45.631294 20131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:45.631290 20134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:45.631314 20132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:07:45.631603 20125 server_base.cc:1034] running on GCE node
I20260925 22:07:45.631793 20125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:07:45.632098 20125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:07:45.633265 20125 hybrid_clock.cc:648] HybridClock initialized: now 1790374065633250 us; error 27 us; skew 500 ppm
Sep 25 22:07:45 dist-test-slave-7mqr krb5kdc[18822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374065, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:07:45.635632 20125 init.cc:377] Logged in from keytab as kudu/127.5.0.66@KRBTEST.COM (short username kudu)
I20260925 22:07:45.636157 20125 webserver.cc:460] Webserver started at http://127.5.0.66:41497/ using document root <none> and password file <none>
I20260925 22:07:45.636384 20125 fs_manager.cc:359] Metadata directory not provided
I20260925 22:07:45.636453 20125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:07:45.637763 20125 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:07:45.638386 20141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:45.638547 20125 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:07:45.638644 20125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/wal
uuid: "6de12817f7c94bdeb7e11241079e9778"
format_stamp: "Formatted at 2026-09-25 22:07:44 on dist-test-slave-7mqr"
server_key: "c1b19e914f4965384d2938c44f636007"
server_key_iv: "a4e7c1b74735afe4fa4291f3d9c0e614"
server_key_version: "encryptionkey@0"
I20260925 22:07:45.639003 20125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:07:45.656466 20125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:07:45.658269 20125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:07:45.658437 20125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:07:45.658675 20125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:07:45.659091 20148 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 22:07:45.660581 20125 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 22:07:45.660645 20125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260925 22:07:45.660687 20125 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 22:07:45.661351 20125 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 22:07:45.661407 20125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260925 22:07:45.661474 20148 tablet_bootstrap.cc:492] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778: Bootstrap starting.
I20260925 22:07:45.668869 20125 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.66:44715
I20260925 22:07:45.668933 20255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.66:44715 every 8 connection(s)
I20260925 22:07:45.669271 20125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
I20260925 22:07:45.670388 20258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:45.669929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.66:51005)
0925 22:07:45.670105 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:45.670211 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":71,"threads_started":1}
I20260925 22:07:45.670634 20148 log.cc:824] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778: Log is configured to *not* fsync() on all Append() calls
W20260925 22:07:45.670668 20256 heartbeater.cc:644] Failed to heartbeat to 127.5.0.126:34019 (0 consecutive failures): Network error: Failed to ping master at 127.5.0.126:34019: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
I20260925 22:07:45.670956 20258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:45.670844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.66:36379)
0925 22:07:45.670897 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:45.670929 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260925 22:07:45.671334 20258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:45.671232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.66:46919)
0925 22:07:45.671283 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:45.671311 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 22:07:45.671514 20256 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 22:07:45.672957 20148 tablet_bootstrap.cc:492] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 22:07:45.673238 20148 tablet_bootstrap.cc:492] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778: Bootstrap complete.
I20260925 22:07:45.674186 20148 ts_tablet_manager.cc:1397] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.007s
I20260925 22:07:45.675365 20148 raft_consensus.cc:348] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcb501810f564e5eb3dadc544cc81b81" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33761 } } peers { permanent_uuid: "6de12817f7c94bdeb7e11241079e9778" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44715 } } peers { permanent_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45619 } }
I20260925 22:07:45.675606 20148 raft_consensus.cc:729] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6de12817f7c94bdeb7e11241079e9778, State: Initialized, Role: FOLLOWER
I20260925 22:07:45.675763 20148 consensus_queue.cc:260] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcb501810f564e5eb3dadc544cc81b81" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33761 } } peers { permanent_uuid: "6de12817f7c94bdeb7e11241079e9778" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44715 } } peers { permanent_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45619 } }
I20260925 22:07:45.676055 20148 ts_tablet_manager.cc:1428] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20260925 22:07:45.676059 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 20125
I20260925 22:07:45.676276 20258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:45.676186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.66:58265)
0925 22:07:45.676216 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:45.676248 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260925 22:07:45.676303 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.67:33761
--local_ip_for_outbound_sockets=127.5.0.67
--tserver_master_addrs=127.5.0.126:34019
--webserver_port=33617
--webserver_interface=127.5.0.67
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:34219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:07:45.676633 20258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:45.676529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.66:39533)
0925 22:07:45.676580 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:45.676608 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260925 22:07:45.757630 20261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:07:45.757823 20261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:07:45.757845 20261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:07:45.758991 20261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:07:45.759033 20261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:07:45.759058 20261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.67
I20260925 22:07:45.760567 20261 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:34219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.67:33761
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.0.67
--webserver_port=33617
--tserver_master_addrs=127.5.0.126:34019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.67
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:07:45.760826 20261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:07:45.761058 20261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:07:45.765980 20269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:45.766016 20266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:07:45.766016 20267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:07:45.766032 20261 server_base.cc:1034] running on GCE node
I20260925 22:07:45.766374 20261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:07:45.766667 20261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:07:45.767844 20261 hybrid_clock.cc:648] HybridClock initialized: now 1790374065767831 us; error 28 us; skew 500 ppm
Sep 25 22:07:45 dist-test-slave-7mqr krb5kdc[18822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374065, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:07:45.770354 20261 init.cc:377] Logged in from keytab as kudu/127.5.0.67@KRBTEST.COM (short username kudu)
I20260925 22:07:45.770867 20261 webserver.cc:460] Webserver started at http://127.5.0.67:33617/ using document root <none> and password file <none>
I20260925 22:07:45.771126 20261 fs_manager.cc:359] Metadata directory not provided
I20260925 22:07:45.771195 20261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:07:45.772598 20261 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:07:45.773161 20276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:45.773326 20261 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:45.773430 20261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/wal
uuid: "dcb501810f564e5eb3dadc544cc81b81"
format_stamp: "Formatted at 2026-09-25 22:07:44 on dist-test-slave-7mqr"
server_key: "4c4f95fcc0c5436faecefcf7ff29589b"
server_key_iv: "34ccab7f0deca77aa655067ee11d3747"
server_key_version: "encryptionkey@0"
I20260925 22:07:45.773772 20261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:07:45.784803 20261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:07:45.786604 20261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:07:45.786763 20261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:07:45.786999 20261 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:07:45.787304 20261 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:07:45.787364 20261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:45.787405 20261 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:07:45.787432 20261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:07:45.793426 20261 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.67:33761
I20260925 22:07:45.793483 20389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.67:33761 every 8 connection(s)
I20260925 22:07:45.793807 20261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
I20260925 22:07:45.794771 20392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:45.794409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.67:50913)
0925 22:07:45.794556 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:45.794631 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
W20260925 22:07:45.795176 20390 heartbeater.cc:644] Failed to heartbeat to 127.5.0.126:34019 (0 consecutive failures): Network error: Failed to ping master at 127.5.0.126:34019: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
I20260925 22:07:45.795462 20392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:45.795375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.67:59961)
0925 22:07:45.795419 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:45.795446 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260925 22:07:45.795805 20392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:45.795716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.67:51743)
0925 22:07:45.795771 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:45.795792 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260925 22:07:45.795944 20390 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 22:07:45.801095 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 20261
I20260925 22:07:46.589324 20394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:46.589075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.65:50289)
0925 22:07:46.589193 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:46.589250 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":61,"threads_started":1}
I20260925 22:07:46.677239 20395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:46.677037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.66:41443)
0925 22:07:46.677151 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:46.677182 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":73,"threads_started":1}
I20260925 22:07:46.796452 20396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:46.796257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.67:54775)
0925 22:07:46.796370 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:46.796405 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":60,"threads_started":1}
I20260925 22:07:46.910378 20397 raft_consensus.cc:482] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:07:46.910499 20397 raft_consensus.cc:504] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcb501810f564e5eb3dadc544cc81b81" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33761 } } peers { permanent_uuid: "6de12817f7c94bdeb7e11241079e9778" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44715 } } peers { permanent_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45619 } }
I20260925 22:07:46.910816 20397 leader_election.cc:288] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dcb501810f564e5eb3dadc544cc81b81 (127.5.0.67:33761), ea2086a88bfb40c7afe2181e79e15bf1 (127.5.0.65:45619)
Sep 25 22:07:46 dist-test-slave-7mqr krb5kdc[18822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374065, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for kudu/127.5.0.65@KRBTEST.COM
Sep 25 22:07:46 dist-test-slave-7mqr krb5kdc[18822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374065, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for kudu/127.5.0.67@KRBTEST.COM
I20260925 22:07:46.928337 20398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:46.911021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:45619 (local address 127.5.0.66:39723)
0925 22:07:46.911176 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:46.911187 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:46.911306 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:46.911627 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:46.911633 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:46.912082 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:07:46.912387 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:46.912393 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:46.913626 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:46.913627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:46.914451 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:46.914453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:46.915042 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:46.915043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:46.915171 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:46.921832 (+ 6661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:07:46.921842 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:07:46.923982 (+ 2140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:07:46.926692 (+ 2710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:46.926698 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:46.926700 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:46.926919 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:46.927209 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:46.927210 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:46.927210 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:46.927319 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:46.927722 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:07:46.927729 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:07:46.927865 (+ 136us) client_negotiation.cc:769] Sending connection context
0925 22:07:46.928009 (+ 144us) client_negotiation.cc:240] Negotiation successful
0925 22:07:46.928204 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":34,"thread_start_us":67,"threads_started":1}
I20260925 22:07:46.928404 20399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:46.911116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:39723 (local address 127.5.0.65:45619)
0925 22:07:46.911259 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:46.911265 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:46.911278 (+ 13us) server_negotiation.cc:407] Connection header received
0925 22:07:46.911463 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:46.911465 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:46.911494 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:46.911562 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:46.912480 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:46.913532 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:46.914542 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:46.914946 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:46.915048 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:46.924178 (+ 9130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:46.924184 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:46.924184 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:46.924199 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:46.926586 (+ 2387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:46.927023 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:46.927028 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:46.927028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:46.927128 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:46.927391 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:46.927392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:46.927392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:46.927646 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:46.927743 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:46.928079 (+ 336us) server_negotiation.cc:299] Negotiation successful
0925 22:07:46.928261 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":44,"threads_started":1}
I20260925 22:07:46.928485 20400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:46.911184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:53389 (local address 127.5.0.67:33761)
0925 22:07:46.911316 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:46.911326 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:46.911362 (+ 36us) server_negotiation.cc:407] Connection header received
0925 22:07:46.912032 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:46.912035 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:46.912085 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:46.912217 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:46.912558 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:46.913557 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:46.914599 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:46.914965 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:46.915096 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:46.924593 (+ 9497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:46.924597 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:46.924597 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:46.924609 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:46.926876 (+ 2267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:46.927270 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:46.927273 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:46.927273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:46.927376 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:46.927615 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:46.927616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:46.927616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:46.927862 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:46.928021 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:46.928232 (+ 211us) server_negotiation.cc:299] Negotiation successful
0925 22:07:46.928363 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":69,"threads_started":1}
I20260925 22:07:46.928383 20395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:46.910968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:33761 (local address 127.5.0.66:53389)
0925 22:07:46.911047 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:46.911075 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:46.911270 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:46.912217 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:46.912222 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:46.912391 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:07:46.912448 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:46.912450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:46.913666 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:46.913667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:46.914449 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:46.914451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:46.915067 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:46.915068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:46.915170 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:46.921847 (+ 6677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:07:46.921852 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:07:46.924381 (+ 2529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:07:46.926972 (+ 2591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:46.926973 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:46.926974 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:46.927164 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:46.927456 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:46.927457 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:46.927458 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:46.927532 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:46.928022 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:07:46.928023 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:07:46.928072 (+ 49us) client_negotiation.cc:769] Sending connection context
0925 22:07:46.928141 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 22:07:46.928254 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":74}
I20260925 22:07:46.929185 20075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2c88155a71a4eef8649df0ffaa9d356" candidate_uuid: "6de12817f7c94bdeb7e11241079e9778" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" is_pre_election: true
I20260925 22:07:46.929349 20075 raft_consensus.cc:2454] T b2c88155a71a4eef8649df0ffaa9d356 P ea2086a88bfb40c7afe2181e79e15bf1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6de12817f7c94bdeb7e11241079e9778 in term 1.
W20260925 22:07:46.929495 20142 leader_election.cc:341] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dcb501810f564e5eb3dadc544cc81b81 (127.5.0.67:33761): Not found: Tablet not found: b2c88155a71a4eef8649df0ffaa9d356
I20260925 22:07:46.929630 20144 leader_election.cc:302] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6de12817f7c94bdeb7e11241079e9778, ea2086a88bfb40c7afe2181e79e15bf1; no voters: dcb501810f564e5eb3dadc544cc81b81
I20260925 22:07:46.929858 20397 raft_consensus.cc:2781] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 22:07:46.929935 20397 raft_consensus.cc:482] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:07:46.929970 20397 raft_consensus.cc:3037] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [term 1 FOLLOWER]: Advancing to term 2
I20260925 22:07:46.931094 20397 raft_consensus.cc:504] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcb501810f564e5eb3dadc544cc81b81" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33761 } } peers { permanent_uuid: "6de12817f7c94bdeb7e11241079e9778" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44715 } } peers { permanent_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45619 } }
I20260925 22:07:46.931252 20397 leader_election.cc:288] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [CANDIDATE]: Term 2 election: Requested vote from peers dcb501810f564e5eb3dadc544cc81b81 (127.5.0.67:33761), ea2086a88bfb40c7afe2181e79e15bf1 (127.5.0.65:45619)
I20260925 22:07:46.931409 20075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2c88155a71a4eef8649df0ffaa9d356" candidate_uuid: "6de12817f7c94bdeb7e11241079e9778" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "ea2086a88bfb40c7afe2181e79e15bf1"
I20260925 22:07:46.931526 20075 raft_consensus.cc:3037] T b2c88155a71a4eef8649df0ffaa9d356 P ea2086a88bfb40c7afe2181e79e15bf1 [term 1 FOLLOWER]: Advancing to term 2
W20260925 22:07:46.931566 20142 leader_election.cc:341] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer dcb501810f564e5eb3dadc544cc81b81 (127.5.0.67:33761): Not found: Tablet not found: b2c88155a71a4eef8649df0ffaa9d356
I20260925 22:07:46.932682 20075 raft_consensus.cc:2454] T b2c88155a71a4eef8649df0ffaa9d356 P ea2086a88bfb40c7afe2181e79e15bf1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6de12817f7c94bdeb7e11241079e9778 in term 2.
I20260925 22:07:46.932893 20144 leader_election.cc:302] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6de12817f7c94bdeb7e11241079e9778, ea2086a88bfb40c7afe2181e79e15bf1; no voters: dcb501810f564e5eb3dadc544cc81b81
I20260925 22:07:46.933032 20397 raft_consensus.cc:2781] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 22:07:46.933178 20397 raft_consensus.cc:686] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [term 2 LEADER]: Becoming Leader. State: Replica: 6de12817f7c94bdeb7e11241079e9778, State: Running, Role: LEADER
I20260925 22:07:46.933275 20397 consensus_queue.cc:237] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcb501810f564e5eb3dadc544cc81b81" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33761 } } peers { permanent_uuid: "6de12817f7c94bdeb7e11241079e9778" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44715 } } peers { permanent_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45619 } }
I20260925 22:07:46.933542 20395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:46.933408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.66:53529)
0925 22:07:46.933447 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:46.933514 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260925 22:07:47.373705 20142 consensus_peers.cc:487] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 -> Peer dcb501810f564e5eb3dadc544cc81b81 (127.5.0.67:33761): Couldn't send request to peer dcb501810f564e5eb3dadc544cc81b81. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b2c88155a71a4eef8649df0ffaa9d356. This is attempt 1: this message will repeat every 5th retry.
I20260925 22:07:47.533667 20075 raft_consensus.cc:1261] T b2c88155a71a4eef8649df0ffaa9d356 P ea2086a88bfb40c7afe2181e79e15bf1 [term 2 FOLLOWER]: Refusing update from remote peer 6de12817f7c94bdeb7e11241079e9778: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260925 22:07:47.534060 20397 consensus_queue.cc:1035] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260925 22:07:47.534933 20405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:47.534768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.65:59477)
0925 22:07:47.534841 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:47.534888 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59,"thread_start_us":45,"threads_started":1}
I20260925 22:07:47.535871 20409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:47.535682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.66:37951)
0925 22:07:47.535778 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:47.535812 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70,"thread_start_us":53,"threads_started":1}
I20260925 22:07:47.535943 20405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:47.535799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.65:34283)
0925 22:07:47.535855 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:47.535886 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260925 22:07:47.797051 20410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:47.796848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.67:35231)
0925 22:07:47.796964 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:47.797003 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":58,"threads_started":1}
I20260925 22:07:47.928984 20411 ts_tablet_manager.cc:927] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81: Initiating tablet copy from peer 6de12817f7c94bdeb7e11241079e9778 (127.5.0.66:44715)
I20260925 22:07:47.929255 20411 tablet_copy_client.cc:316] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81: tablet copy: Beginning tablet copy session from remote peer at address 127.5.0.66:44715
Sep 25 22:07:47 dist-test-slave-7mqr krb5kdc[18822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374065, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for kudu/127.5.0.66@KRBTEST.COM
I20260925 22:07:47.937186 20410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:47.929574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:44715 (local address 127.5.0.67:56711)
0925 22:07:47.929653 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:47.929672 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 22:07:47.929752 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:07:47.930028 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:07:47.930035 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:07:47.930316 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:07:47.930388 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:47.930391 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:47.931225 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:47.931226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:47.931803 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:07:47.931804 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:07:47.932309 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:07:47.932310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:07:47.932374 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:47.932971 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:07:47.932976 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:07:47.934429 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:07:47.936171 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:47.936178 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:47.936180 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:47.936383 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:47.936576 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:07:47.936577 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:07:47.936577 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:07:47.936635 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:07:47.936886 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:07:47.936892 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:07:47.936939 (+ 47us) client_negotiation.cc:769] Sending connection context
0925 22:07:47.937011 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 22:07:47.937100 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260925 22:07:47.937199 20413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:47.929666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:56711 (local address 127.5.0.66:44715)
0925 22:07:47.929836 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 22:07:47.929841 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:07:47.929851 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:07:47.929875 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:07:47.929876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:07:47.929889 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:07:47.929954 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:07:47.930466 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:47.931148 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:47.931877 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:07:47.932237 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:07:47.932294 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:07:47.934516 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:07:47.934526 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:07:47.934526 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:07:47.934534 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:07:47.936099 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:47.936457 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:47.936459 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:47.936459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:47.936516 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:07:47.936688 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:07:47.936688 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:07:47.936688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:07:47.936824 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:07:47.936885 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 22:07:47.937064 (+ 179us) server_negotiation.cc:299] Negotiation successful
0925 22:07:47.937131 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":94,"threads_started":1}
I20260925 22:07:47.937670 20230 tablet_copy_service.cc:140] P 6de12817f7c94bdeb7e11241079e9778: Received BeginTabletCopySession request for tablet b2c88155a71a4eef8649df0ffaa9d356 from peer dcb501810f564e5eb3dadc544cc81b81 ({username='kudu', principal='kudu/127.5.0.67@KRBTEST.COM'} at 127.5.0.67:56711)
I20260925 22:07:47.937732 20230 tablet_copy_service.cc:161] P 6de12817f7c94bdeb7e11241079e9778: Beginning new tablet copy session on tablet b2c88155a71a4eef8649df0ffaa9d356 from peer dcb501810f564e5eb3dadc544cc81b81 at {username='kudu', principal='kudu/127.5.0.67@KRBTEST.COM'} at 127.5.0.67:56711: session id = dcb501810f564e5eb3dadc544cc81b81-b2c88155a71a4eef8649df0ffaa9d356
I20260925 22:07:47.938179 20230 tablet_copy_source_session.cc:205] T b2c88155a71a4eef8649df0ffaa9d356 P 6de12817f7c94bdeb7e11241079e9778: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 22:07:47.939117 20411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2c88155a71a4eef8649df0ffaa9d356. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:07:47.941902 20411 tablet_copy_client.cc:791] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81: tablet copy: Starting download of 0 data blocks...
I20260925 22:07:47.942082 20411 tablet_copy_client.cc:655] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81: tablet copy: Starting download of 1 WAL segments...
I20260925 22:07:47.942981 20411 tablet_copy_client.cc:531] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 22:07:47.943640 20411 tablet_bootstrap.cc:492] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81: Bootstrap starting.
I20260925 22:07:47.944964 20411 log.cc:824] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81: Log is configured to *not* fsync() on all Append() calls
I20260925 22:07:47.947245 20411 tablet_bootstrap.cc:492] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 22:07:47.947472 20411 tablet_bootstrap.cc:492] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81: Bootstrap complete.
I20260925 22:07:47.947614 20411 ts_tablet_manager.cc:1397] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260925 22:07:47.948612 20411 raft_consensus.cc:348] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcb501810f564e5eb3dadc544cc81b81" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33761 } } peers { permanent_uuid: "6de12817f7c94bdeb7e11241079e9778" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44715 } } peers { permanent_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45619 } }
I20260925 22:07:47.948788 20411 raft_consensus.cc:729] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcb501810f564e5eb3dadc544cc81b81, State: Initialized, Role: FOLLOWER
I20260925 22:07:47.948942 20411 consensus_queue.cc:260] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcb501810f564e5eb3dadc544cc81b81" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33761 } } peers { permanent_uuid: "6de12817f7c94bdeb7e11241079e9778" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 44715 } } peers { permanent_uuid: "ea2086a88bfb40c7afe2181e79e15bf1" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 45619 } }
I20260925 22:07:47.949190 20411 ts_tablet_manager.cc:1428] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
I20260925 22:07:47.949478 20230 tablet_copy_service.cc:342] P 6de12817f7c94bdeb7e11241079e9778: Request end of tablet copy session dcb501810f564e5eb3dadc544cc81b81-b2c88155a71a4eef8649df0ffaa9d356 received from {username='kudu', principal='kudu/127.5.0.67@KRBTEST.COM'} at 127.5.0.67:56711
I20260925 22:07:47.949564 20230 tablet_copy_service.cc:433] P 6de12817f7c94bdeb7e11241079e9778: ending tablet copy session dcb501810f564e5eb3dadc544cc81b81-b2c88155a71a4eef8649df0ffaa9d356 on tablet b2c88155a71a4eef8649df0ffaa9d356 with peer dcb501810f564e5eb3dadc544cc81b81
I20260925 22:07:47.949558 20410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:47.949404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.67:55101)
0925 22:07:47.949455 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:47.949526 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260925 22:07:47.949869 20410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:47.949785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.67:43393)
0925 22:07:47.949820 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:47.949842 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260925 22:07:48.315958 20418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:48.315671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.66:47405)
0925 22:07:48.315819 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:48.315865 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":92,"threads_started":1}
I20260925 22:07:48.315951 20344 raft_consensus.cc:1203] T b2c88155a71a4eef8649df0ffaa9d356 P dcb501810f564e5eb3dadc544cc81b81 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260925 22:07:48.316421 20410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:48.316310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.67:36817)
0925 22:07:48.316360 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:48.316389 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260925 22:07:48.536525 20419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:48.536330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.65:47057)
0925 22:07:48.536447 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:48.536482 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260925 22:07:49.316613 20420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:49.316389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.66:57513)
0925 22:07:49.316515 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:49.316567 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":58,"threads_started":1}
I20260925 22:07:49.316985 20421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:49.316803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.67:33977)
0925 22:07:49.316913 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:49.316948 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":67,"threads_started":1}
I20260925 22:07:49.537236 20422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:49.537019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.65:35385)
0925 22:07:49.537163 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:49.537200 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":72,"threads_started":1}
I20260925 22:07:50.317328 20423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:50.317083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.66:37095)
0925 22:07:50.317222 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:50.317260 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":57,"threads_started":1}
I20260925 22:07:50.317651 20424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:50.317386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.67:50535)
0925 22:07:50.317545 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:50.317575 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":56,"threads_started":1}
I20260925 22:07:50.538007 20425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:50.537808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.65:34873)
0925 22:07:50.537932 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:50.537971 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":60,"threads_started":1}
I20260925 22:07:51.318055 20426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:51.317830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.66:52543)
0925 22:07:51.317964 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:51.317995 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
I20260925 22:07:51.318234 20427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:51.318051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.67:51275)
0925 22:07:51.318157 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:51.318190 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":80,"threads_started":1}
I20260925 22:07:51.318481 20426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:51.318399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.66:60025)
0925 22:07:51.318431 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:51.318456 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260925 22:07:51.538609 20428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:51.538430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.65:49471)
0925 22:07:51.538543 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:51.538573 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":58,"threads_started":1}
I20260925 22:07:52.318967 20429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:52.318708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.67:37701)
0925 22:07:52.318863 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:52.318908 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":73,"threads_started":1}
I20260925 22:07:52.319182 20430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:52.318939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.66:33445)
0925 22:07:52.319093 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:52.319136 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":63,"threads_started":1}
I20260925 22:07:52.539326 20431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:52.539126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.65:55129)
0925 22:07:52.539257 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:52.539290 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":72,"threads_started":1}
I20260925 22:07:53.319738 20432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:53.319443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.67:53895)
0925 22:07:53.319643 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:53.319680 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":127,"threads_started":1}
I20260925 22:07:53.319823 20433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:53.319631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.66:55615)
0925 22:07:53.319756 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:53.319784 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":74,"threads_started":1}
I20260925 22:07:53.540047 20434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:53.539808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.65:58437)
0925 22:07:53.539962 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:53.540008 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":124,"threads_started":1}
I20260925 22:07:54.320485 20436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:54.320271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.66:55449)
0925 22:07:54.320414 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:54.320447 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":85,"threads_started":1}
I20260925 22:07:54.320479 20435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:54.320249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.67:52915)
0925 22:07:54.320385 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:54.320432 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":58,"threads_started":1}
I20260925 22:07:54.540726 20437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:54.540520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.65:43403)
0925 22:07:54.540655 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:54.540690 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":77,"threads_started":1}
I20260925 22:07:55.321125 20439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:55.320904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.67:44395)
0925 22:07:55.321040 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:55.321078 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":82,"threads_started":1}
I20260925 22:07:55.321125 20438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:55.320892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.66:60699)
0925 22:07:55.321057 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:55.321082 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":97,"threads_started":1}
I20260925 22:07:55.541385 20440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:55.541125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.65:46951)
0925 22:07:55.541303 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:55.541344 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":80,"threads_started":1}
I20260925 22:07:56.321884 20442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:56.321651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.66:47533)
0925 22:07:56.321797 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:56.321835 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":79,"threads_started":1}
I20260925 22:07:56.321897 20441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:56.321652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.67:44375)
0925 22:07:56.321824 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:56.321856 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":82,"threads_started":1}
I20260925 22:07:56.542105 20443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:56.541859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.65:39221)
0925 22:07:56.542031 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:56.542064 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":67,"threads_started":1}
I20260925 22:07:57.322629 20445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:57.322403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.66:54729)
0925 22:07:57.322556 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:57.322590 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":104,"threads_started":1}
I20260925 22:07:57.322628 20444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:57.322401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.67:41979)
0925 22:07:57.322548 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:57.322581 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
I20260925 22:07:57.542881 20446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:57.542683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.65:40325)
0925 22:07:57.542806 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:57.542841 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":67,"threads_started":1}
I20260925 22:07:58.323331 20448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:58.323111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.67:50347)
0925 22:07:58.323256 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:58.323283 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":71,"threads_started":1}
I20260925 22:07:58.323355 20449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:58.323111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.66:47435)
0925 22:07:58.323250 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:58.323313 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":68,"threads_started":1}
I20260925 22:07:58.543588 20451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:58.543346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.65:47893)
0925 22:07:58.543508 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:58.543544 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":75,"threads_started":1}
I20260925 22:07:59.324146 20452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:59.323854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.66:46763)
0925 22:07:59.324052 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:59.324098 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":144,"threads_started":1}
I20260925 22:07:59.324149 20453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:59.323942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.67:33939)
0925 22:07:59.324069 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:59.324109 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":75,"threads_started":1}
I20260925 22:07:59.544330 20454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:07:59.544113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.65:37745)
0925 22:07:59.544251 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0925 22:07:59.544284 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
I20260925 22:08:00.324833 20456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:00.324635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.66:37245)
0925 22:08:00.324761 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:00.324791 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":74,"threads_started":1}
I20260925 22:08:00.324831 20455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:00.324612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.67:40663)
0925 22:08:00.324747 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:00.324784 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
I20260925 22:08:00.545153 20457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:00.544881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34019 (local address 127.5.0.65:49197)
0925 22:08:00.545058 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:00.545103 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:34019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":66,"threads_started":1}
I20260925 22:08:00.812592 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 19962
I20260925 22:08:00.817317 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 20125
I20260925 22:08:00.821869 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 20261
2026-09-25T22:08:00Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19038 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:08:01 dist-test-slave-7mqr krb5kdc[20460](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:08:01 dist-test-slave-7mqr krb5kdc[20460](info): set up 2 sockets
Sep 25 22:08:01 dist-test-slave-7mqr krb5kdc[20460](info): commencing operation
krb5kdc: starting...
W20260925 22:08:02.564397 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:08:02 dist-test-slave-7mqr krb5kdc[20460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:08:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:08:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.0.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
I20260925 22:08:02.680787 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:42695
--webserver_interface=127.5.0.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:37557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:42695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:08:02.760842 20473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:08:02.761099 20473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:08:02.761138 20473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:08:02.762269 20473 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 22:08:02.762336 20473 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 22:08:02.762362 20473 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 22:08:02.762392 20473 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:08:02.762418 20473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:08:02.762449 20473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:08:02.762477 20473 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:08:02.762503 20473 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:08:02.764046 20473 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:37557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:42695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:42695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:08:02.764326 20473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:08:02.764560 20473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:08:02.769325 20481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:08:02.769343 20479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:08:02.769428 20478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:08:02.769565 20473 server_base.cc:1034] running on GCE node
I20260925 22:08:02.769845 20473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:08:02.770172 20473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:08:02.771332 20473 hybrid_clock.cc:648] HybridClock initialized: now 1790374082771308 us; error 41 us; skew 500 ppm
Sep 25 22:08:02 dist-test-slave-7mqr krb5kdc[20460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374082, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:08:02.773839 20473 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:08:02.774235 20473 webserver.cc:460] Webserver started at http://127.5.0.126:37595/ using document root <none> and password file <none>
I20260925 22:08:02.774415 20473 fs_manager.cc:359] Metadata directory not provided
I20260925 22:08:02.774454 20473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:08:02.774585 20473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:08:02.775635 20473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "521bc070528545ebb74dce96dfc41a2d"
format_stamp: "Formatted at 2026-09-25 22:08:02 on dist-test-slave-7mqr"
server_key: "0ad726727fd5c79765dbef07dc511f23"
server_key_iv: "166fd940d02c1fc69bd05205333f223b"
server_key_version: "encryptionkey@0"
I20260925 22:08:02.776018 20473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "521bc070528545ebb74dce96dfc41a2d"
format_stamp: "Formatted at 2026-09-25 22:08:02 on dist-test-slave-7mqr"
server_key: "0ad726727fd5c79765dbef07dc511f23"
server_key_iv: "166fd940d02c1fc69bd05205333f223b"
server_key_version: "encryptionkey@0"
I20260925 22:08:02.777325 20473 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:08:02.778079 20488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:08:02.778280 20473 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:08:02.778391 20473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "521bc070528545ebb74dce96dfc41a2d"
format_stamp: "Formatted at 2026-09-25 22:08:02 on dist-test-slave-7mqr"
server_key: "0ad726727fd5c79765dbef07dc511f23"
server_key_iv: "166fd940d02c1fc69bd05205333f223b"
server_key_version: "encryptionkey@0"
I20260925 22:08:02.778471 20473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:08:02.790486 20473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:08:02.792809 20473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:08:02.792977 20473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:08:02.796940 20473 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:42695
I20260925 22:08:02.796976 20540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:42695 every 8 connection(s)
I20260925 22:08:02.797344 20473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:08:02.798228 20541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:08:02.800570 20541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d: Bootstrap starting.
I20260925 22:08:02.801175 20541 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d: Neither blocks nor log segments found. Creating new log.
I20260925 22:08:02.801471 20541 log.cc:824] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d: Log is configured to *not* fsync() on all Append() calls
I20260925 22:08:02.802170 20541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d: No bootstrap required, opened a new log
I20260925 22:08:02.803732 20541 raft_consensus.cc:348] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521bc070528545ebb74dce96dfc41a2d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42695 } }
I20260925 22:08:02.803869 20541 raft_consensus.cc:374] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:08:02.803941 20541 raft_consensus.cc:729] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 521bc070528545ebb74dce96dfc41a2d, State: Initialized, Role: FOLLOWER
I20260925 22:08:02.804103 20541 consensus_queue.cc:260] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521bc070528545ebb74dce96dfc41a2d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42695 } }
I20260925 22:08:02.804169 20541 raft_consensus.cc:388] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:08:02.804191 20541 raft_consensus.cc:482] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:08:02.804261 20541 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:08:02.804773 20541 raft_consensus.cc:504] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521bc070528545ebb74dce96dfc41a2d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42695 } }
I20260925 22:08:02.804883 20541 leader_election.cc:302] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 521bc070528545ebb74dce96dfc41a2d; no voters:
I20260925 22:08:02.805052 20541 leader_election.cc:288] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:08:02.805105 20544 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:08:02.805141 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 20473
I20260925 22:08:02.805246 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:08:02.805262 20544 raft_consensus.cc:686] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [term 1 LEADER]: Becoming Leader. State: Replica: 521bc070528545ebb74dce96dfc41a2d, State: Running, Role: LEADER
I20260925 22:08:02.805332 20541 sys_catalog.cc:564] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:08:02.805454 5121 external_mini_cluster.cc:1428] Setting key 20fd0c5855ffedbd4ff1c52df67b3509
I20260925 22:08:02.805346 20544 consensus_queue.cc:237] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521bc070528545ebb74dce96dfc41a2d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42695 } }
I20260925 22:08:02.806123 20546 sys_catalog.cc:455] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "521bc070528545ebb74dce96dfc41a2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521bc070528545ebb74dce96dfc41a2d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42695 } } }
I20260925 22:08:02.806217 20546 sys_catalog.cc:458] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [sys.catalog]: This master's current role is: LEADER
I20260925 22:08:02.806624 20545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 521bc070528545ebb74dce96dfc41a2d. Latest consensus state: current_term: 1 leader_uuid: "521bc070528545ebb74dce96dfc41a2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521bc070528545ebb74dce96dfc41a2d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42695 } } }
I20260925 22:08:02.806746 20545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [sys.catalog]: This master's current role is: LEADER
I20260925 22:08:02.806846 20555 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:08:02.807519 20555 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:08:02.809058 20555 catalog_manager.cc:1347] Generated new cluster ID: 92e572e87e274bee8b613af0ca1dfb9e
I20260925 22:08:02.809109 20555 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:08:02 dist-test-slave-7mqr krb5kdc[20460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:08:02.817440 20551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:02.806013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40502 (local address 127.5.0.126:42695)
0925 22:08:02.806427 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0925 22:08:02.806439 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 22:08:02.806474 (+ 35us) server_negotiation.cc:407] Connection header received
0925 22:08:02.806602 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:08:02.806610 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:08:02.806675 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:08:02.806906 (+ 231us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:08:02.807834 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:02.809190 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:02.810261 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:02.810889 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:02.811004 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:02.813807 (+ 2803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:08:02.813811 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:08:02.813812 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:08:02.813823 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:08:02.815721 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:08:02.816139 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:08:02.816142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:08:02.816143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:08:02.816252 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:08:02.816479 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:08:02.816481 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:08:02.816481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:08:02.816816 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:08:02.816963 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0925 22:08:02.817088 (+ 125us) server_negotiation.cc:299] Negotiation successful
0925 22:08:02.817224 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":99,"threads_started":1}
I20260925 22:08:02.847389 20555 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:08:02.848163 20555 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:08:02.854135 20555 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d: Generated new TSK 0
I20260925 22:08:02.854362 20555 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.0.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
I20260925 22:08:02.882273 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.65:0
--local_ip_for_outbound_sockets=127.5.0.65
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:42695
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:37557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:08:02.968700 20567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:08:02.968887 20567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:08:02.968910 20567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:08:02.970041 20567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:08:02.970082 20567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:08:02.970134 20567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.65
I20260925 22:08:02.971550 20567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:37557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:42695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.65
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:08:02.971789 20567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:08:02.972030 20567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:08:02.976775 20573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:08:02.976905 20567 server_base.cc:1034] running on GCE node
W20260925 22:08:02.976796 20572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:08:02.976778 20575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:08:02.977217 20567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:08:02.977438 20567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:08:02.978590 20567 hybrid_clock.cc:648] HybridClock initialized: now 1790374082978570 us; error 37 us; skew 500 ppm
Sep 25 22:08:02 dist-test-slave-7mqr krb5kdc[20460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374082, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:08:02.981046 20567 init.cc:377] Logged in from keytab as kudu/127.5.0.65@KRBTEST.COM (short username kudu)
I20260925 22:08:02.981580 20567 webserver.cc:460] Webserver started at http://127.5.0.65:42041/ using document root <none> and password file <none>
I20260925 22:08:02.981801 20567 fs_manager.cc:359] Metadata directory not provided
I20260925 22:08:02.981866 20567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:08:02.982016 20567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:08:02.983085 20567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/data/instance:
uuid: "38b225e2ae0f4f8e971038cc071def22"
format_stamp: "Formatted at 2026-09-25 22:08:02 on dist-test-slave-7mqr"
server_key: "938f1b6a9bedd22127b0db309cfa085d"
server_key_iv: "94b871fada6e304e219c5560df27ead0"
server_key_version: "encryptionkey@0"
I20260925 22:08:02.983450 20567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance:
uuid: "38b225e2ae0f4f8e971038cc071def22"
format_stamp: "Formatted at 2026-09-25 22:08:02 on dist-test-slave-7mqr"
server_key: "938f1b6a9bedd22127b0db309cfa085d"
server_key_iv: "94b871fada6e304e219c5560df27ead0"
server_key_version: "encryptionkey@0"
I20260925 22:08:02.984798 20567 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:08:02.985466 20582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:08:02.985634 20567 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:08:02.985729 20567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/wal
uuid: "38b225e2ae0f4f8e971038cc071def22"
format_stamp: "Formatted at 2026-09-25 22:08:02 on dist-test-slave-7mqr"
server_key: "938f1b6a9bedd22127b0db309cfa085d"
server_key_iv: "94b871fada6e304e219c5560df27ead0"
server_key_version: "encryptionkey@0"
I20260925 22:08:02.985808 20567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:08:03.007797 20567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:08:03.009761 20567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:08:03.009934 20567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:08:03.010174 20567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:08:03.010535 20567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:08:03.010591 20567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:08:03.010643 20567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:08:03.010673 20567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:08:03.016402 20567 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.65:37951
I20260925 22:08:03.016458 20695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.65:37951 every 8 connection(s)
I20260925 22:08:03.016836 20567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
I20260925 22:08:03.017556 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 20567
I20260925 22:08:03.017688 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance
I20260925 22:08:03.017907 5121 external_mini_cluster.cc:1428] Setting key b9a53140b1c7f80b0d9af11ab6d02277
Sep 25 22:08:03 dist-test-slave-7mqr krb5kdc[20460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374082, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.0.66@KRBTEST.COM; defaulting to no policy
I20260925 22:08:03.027494 20551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:03.017556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:54127 (local address 127.5.0.126:42695)
0925 22:08:03.017627 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 22:08:03.017629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:08:03.017811 (+ 182us) server_negotiation.cc:407] Connection header received
0925 22:08:03.018393 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:08:03.018394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:08:03.018407 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:08:03.018455 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:08:03.019450 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.020165 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.021491 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.021837 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.021932 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:03.024406 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:08:03.024408 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:08:03.024408 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:08:03.024415 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:08:03.026160 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:08:03.026620 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:08:03.026621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:08:03.026621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:08:03.026672 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:08:03.026906 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:08:03.026906 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:08:03.026907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:08:03.027004 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:08:03.027055 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 22:08:03.027340 (+ 285us) server_negotiation.cc:299] Negotiation successful
0925 22:08:03.027426 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 22:08:03.027762 20698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:03.017517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.65:54127)
0925 22:08:03.017670 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:03.017711 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 22:08:03.018298 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:08:03.018531 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:08:03.018551 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:08:03.019160 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:08:03.019347 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:03.019351 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.020252 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.020254 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:03.021393 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:03.021396 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.021947 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.021948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:03.022110 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:03.022864 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:08:03.022876 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:08:03.024304 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:08:03.026283 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:08:03.026289 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:08:03.026291 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:08:03.026545 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:08:03.026737 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:08:03.026738 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:08:03.026738 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:08:03.026849 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:08:03.027061 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:08:03.027066 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:08:03.027177 (+ 111us) client_negotiation.cc:769] Sending connection context
0925 22:08:03.027296 (+ 119us) client_negotiation.cc:240] Negotiation successful
0925 22:08:03.027511 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":83,"threads_started":1}
I20260925 22:08:03.028290 20696 heartbeater.cc:347] Connected to a master server at 127.5.0.126:42695
I20260925 22:08:03.028384 20696 heartbeater.cc:464] Registering TS with master...
I20260925 22:08:03.028517 20696 heartbeater.cc:511] Master 127.5.0.126:42695 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.66@KRBTEST.COM" created.
I20260925 22:08:03.029047 20505 ts_manager.cc:194] Registered new tserver with Master: 38b225e2ae0f4f8e971038cc071def22 (127.5.0.65:37951)
I20260925 22:08:03.029666 20505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.65@KRBTEST.COM'} at 127.5.0.65:54127
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
I20260925 22:08:03.042794 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.66:0
--local_ip_for_outbound_sockets=127.5.0.66
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:42695
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:37557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:08:03.122845 20701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:08:03.123034 20701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:08:03.123056 20701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:08:03.124115 20701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:08:03.124150 20701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:08:03.124176 20701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.66
I20260925 22:08:03.125495 20701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:37557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:42695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.66
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:08:03.125672 20701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:08:03.125833 20701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:08:03.130757 20706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:08:03.130786 20709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:08:03.130756 20707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:08:03.131038 20701 server_base.cc:1034] running on GCE node
I20260925 22:08:03.131266 20701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:08:03.131495 20701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:08:03.132648 20701 hybrid_clock.cc:648] HybridClock initialized: now 1790374083132628 us; error 40 us; skew 500 ppm
Sep 25 22:08:03 dist-test-slave-7mqr krb5kdc[20460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374083, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:08:03.135316 20701 init.cc:377] Logged in from keytab as kudu/127.5.0.66@KRBTEST.COM (short username kudu)
I20260925 22:08:03.135784 20701 webserver.cc:460] Webserver started at http://127.5.0.66:45095/ using document root <none> and password file <none>
I20260925 22:08:03.136041 20701 fs_manager.cc:359] Metadata directory not provided
I20260925 22:08:03.136116 20701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:08:03.136251 20701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:08:03.137280 20701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/data/instance:
uuid: "69e6a6a4c260409d945be3ac9784d5e8"
format_stamp: "Formatted at 2026-09-25 22:08:03 on dist-test-slave-7mqr"
server_key: "a0a60a847f3afade7b4469af792b3ab1"
server_key_iv: "90cf14e5e90db0adf0eba136598352d6"
server_key_version: "encryptionkey@0"
I20260925 22:08:03.137624 20701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance:
uuid: "69e6a6a4c260409d945be3ac9784d5e8"
format_stamp: "Formatted at 2026-09-25 22:08:03 on dist-test-slave-7mqr"
server_key: "a0a60a847f3afade7b4469af792b3ab1"
server_key_iv: "90cf14e5e90db0adf0eba136598352d6"
server_key_version: "encryptionkey@0"
I20260925 22:08:03.138887 20701 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:08:03.139595 20716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:08:03.139779 20701 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:08:03.139873 20701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/wal
uuid: "69e6a6a4c260409d945be3ac9784d5e8"
format_stamp: "Formatted at 2026-09-25 22:08:03 on dist-test-slave-7mqr"
server_key: "a0a60a847f3afade7b4469af792b3ab1"
server_key_iv: "90cf14e5e90db0adf0eba136598352d6"
server_key_version: "encryptionkey@0"
I20260925 22:08:03.139957 20701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:08:03.162295 20701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:08:03.164163 20701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:08:03.164335 20701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:08:03.164561 20701 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:08:03.164870 20701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:08:03.164922 20701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:08:03.164963 20701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:08:03.164991 20701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:08:03.170740 20701 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.66:41847
I20260925 22:08:03.170790 20829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.66:41847 every 8 connection(s)
I20260925 22:08:03.171094 20701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
Sep 25 22:08:03 dist-test-slave-7mqr krb5kdc[20460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374083, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:08:03.177644 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 20701
I20260925 22:08:03.177794 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance
I20260925 22:08:03.178022 5121 external_mini_cluster.cc:1428] Setting key 8a8c20ae5510d0f4516e43855301109b
I20260925 22:08:03.180763 20551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:03.171770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:52163 (local address 127.5.0.126:42695)
0925 22:08:03.171812 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 22:08:03.171814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:08:03.172066 (+ 252us) server_negotiation.cc:407] Connection header received
0925 22:08:03.172549 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:08:03.172550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:08:03.172560 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:08:03.172614 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:08:03.173589 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.174088 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.175070 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.175380 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.175433 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:03.177464 (+ 2031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:08:03.177473 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:08:03.177473 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:08:03.177482 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:08:03.179147 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:08:03.179667 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:08:03.179668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:08:03.179668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:08:03.179719 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:08:03.180038 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:08:03.180038 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:08:03.180039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:08:03.180177 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:08:03.180253 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 22:08:03.180584 (+ 331us) server_negotiation.cc:299] Negotiation successful
0925 22:08:03.180672 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 22:08:03.180979 20832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:03.171754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.66:52163)
0925 22:08:03.171944 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:03.171979 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 22:08:03.172438 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:08:03.172668 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:08:03.172671 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:08:03.173223 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:08:03.173508 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:03.173514 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.174167 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.174168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:03.174991 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:03.174994 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.175450 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.175451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:03.175555 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:03.176009 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:08:03.176017 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:08:03.177358 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:08:03.179234 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:08:03.179241 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:08:03.179243 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:08:03.179556 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:08:03.179785 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:08:03.179786 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:08:03.179787 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:08:03.179967 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:08:03.180256 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:08:03.180263 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:08:03.180428 (+ 165us) client_negotiation.cc:769] Sending connection context
0925 22:08:03.180546 (+ 118us) client_negotiation.cc:240] Negotiation successful
0925 22:08:03.180710 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":88,"threads_started":1}
I20260925 22:08:03.181492 20830 heartbeater.cc:347] Connected to a master server at 127.5.0.126:42695
I20260925 22:08:03.181595 20830 heartbeater.cc:464] Registering TS with master...
I20260925 22:08:03.181783 20830 heartbeater.cc:511] Master 127.5.0.126:42695 requested a full tablet report, sending...
I20260925 22:08:03.182363 20505 ts_manager.cc:194] Registered new tserver with Master: 69e6a6a4c260409d945be3ac9784d5e8 (127.5.0.66:41847)
I20260925 22:08:03.183118 20505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.66@KRBTEST.COM'} at 127.5.0.66:52163
WARNING: no policy specified for kudu/127.5.0.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
I20260925 22:08:03.202894 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.67:0
--local_ip_for_outbound_sockets=127.5.0.67
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:42695
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:37557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:08:03.283552 20835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:08:03.283737 20835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:08:03.283759 20835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:08:03.284830 20835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:08:03.284866 20835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:08:03.284891 20835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.67
I20260925 22:08:03.286234 20835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:37557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:42695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.67
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:08:03.286432 20835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:08:03.286598 20835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:08:03.291373 20840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:08:03.291384 20841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:08:03.291433 20843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:08:03.292222 20835 server_base.cc:1034] running on GCE node
I20260925 22:08:03.292475 20835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:08:03.292722 20835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:08:03.293891 20835 hybrid_clock.cc:648] HybridClock initialized: now 1790374083293861 us; error 59 us; skew 500 ppm
Sep 25 22:08:03 dist-test-slave-7mqr krb5kdc[20460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374083, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:08:03.296307 20835 init.cc:377] Logged in from keytab as kudu/127.5.0.67@KRBTEST.COM (short username kudu)
I20260925 22:08:03.296726 20835 webserver.cc:460] Webserver started at http://127.5.0.67:42047/ using document root <none> and password file <none>
I20260925 22:08:03.296986 20835 fs_manager.cc:359] Metadata directory not provided
I20260925 22:08:03.297057 20835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:08:03.297212 20835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:08:03.298240 20835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/data/instance:
uuid: "370bc5fa7b7445ac9d0d063337dc9587"
format_stamp: "Formatted at 2026-09-25 22:08:03 on dist-test-slave-7mqr"
server_key: "c8941ed9c2ddcc439f622304160133a8"
server_key_iv: "4df5d726ec2659b3e105c4747e153c55"
server_key_version: "encryptionkey@0"
I20260925 22:08:03.298601 20835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance:
uuid: "370bc5fa7b7445ac9d0d063337dc9587"
format_stamp: "Formatted at 2026-09-25 22:08:03 on dist-test-slave-7mqr"
server_key: "c8941ed9c2ddcc439f622304160133a8"
server_key_iv: "4df5d726ec2659b3e105c4747e153c55"
server_key_version: "encryptionkey@0"
I20260925 22:08:03.300030 20835 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 22:08:03.300756 20850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:08:03.300923 20835 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:08:03.301047 20835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/wal
uuid: "370bc5fa7b7445ac9d0d063337dc9587"
format_stamp: "Formatted at 2026-09-25 22:08:03 on dist-test-slave-7mqr"
server_key: "c8941ed9c2ddcc439f622304160133a8"
server_key_iv: "4df5d726ec2659b3e105c4747e153c55"
server_key_version: "encryptionkey@0"
I20260925 22:08:03.301122 20835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:08:03.323797 20835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:08:03.325732 20835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:08:03.325910 20835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:08:03.326143 20835 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:08:03.326465 20835 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:08:03.326526 20835 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:08:03.326562 20835 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:08:03.326614 20835 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:08:03.332379 20835 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.67:39851
I20260925 22:08:03.332422 20963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.67:39851 every 8 connection(s)
I20260925 22:08:03.332722 20835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
I20260925 22:08:03.337508 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 20835
I20260925 22:08:03.337608 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance
I20260925 22:08:03.337822 5121 external_mini_cluster.cc:1428] Setting key e2be34f3e8f7e669b548092e3c2b1982
Sep 25 22:08:03 dist-test-slave-7mqr krb5kdc[20460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374083, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:08:03.342137 20551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:03.333414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:40781 (local address 127.5.0.126:42695)
0925 22:08:03.333456 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 22:08:03.333458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:08:03.333689 (+ 231us) server_negotiation.cc:407] Connection header received
0925 22:08:03.334209 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:08:03.334211 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:08:03.334224 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:08:03.334286 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:08:03.335338 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.335838 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.336857 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.337050 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.337145 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:03.339562 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:08:03.339563 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:08:03.339563 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:08:03.339570 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:08:03.341009 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:08:03.341365 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:08:03.341366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:08:03.341366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:08:03.341417 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:08:03.341623 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:08:03.341623 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:08:03.341623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:08:03.341723 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:08:03.341773 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 22:08:03.342001 (+ 228us) server_negotiation.cc:299] Negotiation successful
0925 22:08:03.342062 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 22:08:03.342329 20966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:03.333412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.67:40781)
0925 22:08:03.333576 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:03.333609 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 22:08:03.334106 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:08:03.334366 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:08:03.334370 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:08:03.335027 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:08:03.335226 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:03.335230 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.335945 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.335946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:03.336751 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:03.336753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.337167 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.337168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:03.337298 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:03.337875 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:08:03.337885 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:08:03.339473 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:08:03.341086 (+ 1613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:08:03.341092 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:08:03.341093 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:08:03.341298 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:08:03.341474 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:08:03.341475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:08:03.341475 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:08:03.341565 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:08:03.341778 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:08:03.341781 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:08:03.341889 (+ 108us) client_negotiation.cc:769] Sending connection context
0925 22:08:03.341969 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 22:08:03.342094 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":75,"threads_started":1}
I20260925 22:08:03.342875 20964 heartbeater.cc:347] Connected to a master server at 127.5.0.126:42695
I20260925 22:08:03.342955 20964 heartbeater.cc:464] Registering TS with master...
I20260925 22:08:03.343087 20964 heartbeater.cc:511] Master 127.5.0.126:42695 requested a full tablet report, sending...
I20260925 22:08:03.343420 20505 ts_manager.cc:194] Registered new tserver with Master: 370bc5fa7b7445ac9d0d063337dc9587 (127.5.0.67:39851)
I20260925 22:08:03.343849 20505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.67@KRBTEST.COM'} at 127.5.0.67:40781
I20260925 22:08:03.349548 5121 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:08:03.356511 20551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:03.350825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40506 (local address 127.5.0.126:42695)
0925 22:08:03.350877 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 22:08:03.350879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:08:03.350996 (+ 117us) server_negotiation.cc:407] Connection header received
0925 22:08:03.351035 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:08:03.351036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:08:03.351044 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:08:03.351098 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:08:03.351591 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.352104 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.352768 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.352943 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.352990 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:03.354029 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:08:03.354035 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:08:03.354035 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:08:03.354041 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:08:03.355548 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:08:03.355843 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:08:03.355843 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:08:03.355844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:08:03.355930 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:08:03.356104 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:08:03.356105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:08:03.356105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:08:03.356239 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:08:03.356315 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 22:08:03.356396 (+ 81us) server_negotiation.cc:299] Negotiation successful
0925 22:08:03.356452 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 22:08:03.357990 20505 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40506:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:08:03.359485 20505 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:08:03.366139 20979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:03.361983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:37951 (local address 127.0.0.1:43134)
0925 22:08:03.362209 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:03.362219 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:08:03.362266 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:08:03.362540 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:08:03.362540 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:08:03.362542 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:08:03.362581 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:03.362583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.363560 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.363561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:03.364883 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:03.364886 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.365834 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.365835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:03.365906 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:03.365920 (+ 14us) client_negotiation.cc:769] Sending connection context
0925 22:08:03.365994 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 22:08:03.366048 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":33,"thread_start_us":68,"threads_started":1}
I20260925 22:08:03.366173 20980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:03.362224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43134 (local address 127.5.0.65:37951)
0925 22:08:03.362375 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0925 22:08:03.362379 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:08:03.362388 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:08:03.362426 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:08:03.362429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:08:03.362443 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:08:03.362660 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:08:03.362742 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.363438 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.365002 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.365725 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.365831 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:03.365886 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 22:08:03.366023 (+ 137us) server_negotiation.cc:299] Negotiation successful
0925 22:08:03.366084 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":87,"threads_started":1}
I20260925 22:08:03.366317 20977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:03.361860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44744 (local address 127.5.0.67:39851)
0925 22:08:03.362289 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0925 22:08:03.362294 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:08:03.362302 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:08:03.362338 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:08:03.362340 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:08:03.362353 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:08:03.362429 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:08:03.362605 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.363721 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.365012 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.365723 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.365799 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:03.365860 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 22:08:03.366180 (+ 320us) server_negotiation.cc:299] Negotiation successful
0925 22:08:03.366227 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":98,"threads_started":1}
I20260925 22:08:03.366375 20975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:03.361777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:39851 (local address 127.0.0.1:44744)
0925 22:08:03.362111 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:03.362123 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:08:03.362177 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:08:03.362428 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:08:03.362429 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:08:03.362431 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:08:03.362482 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:03.362484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.363828 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.363829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:03.364936 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:03.364938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.365896 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.365897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:03.366009 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:03.366013 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:08:03.366065 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 22:08:03.366106 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":57,"threads_started":1}
I20260925 22:08:03.367007 20898 tablet_service.cc:1467] Processing CreateTablet for tablet 5867217ab10c4eaa9059a741f8866553 (DEFAULT_TABLE table=test-table [id=f3beaa5d866a4f4e9e28ee4b6be0dcb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:08:03.366990 20630 tablet_service.cc:1467] Processing CreateTablet for tablet 5867217ab10c4eaa9059a741f8866553 (DEFAULT_TABLE table=test-table [id=f3beaa5d866a4f4e9e28ee4b6be0dcb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:08:03.367347 20630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5867217ab10c4eaa9059a741f8866553. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:08:03.367345 20898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5867217ab10c4eaa9059a741f8866553. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:08:03.367363 20976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:03.361772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:41847 (local address 127.0.0.1:33688)
0925 22:08:03.361931 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:03.361971 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 22:08:03.362035 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:08:03.362357 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:08:03.362360 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:08:03.362361 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:08:03.362431 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:03.362433 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.365030 (+ 2597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.365030 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:03.365680 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:03.365681 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.367106 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.367107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:03.367173 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:03.367179 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 22:08:03.367258 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 22:08:03.367301 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":105,"threads_started":1}
I20260925 22:08:03.367393 20978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:03.361960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33688 (local address 127.5.0.66:41847)
0925 22:08:03.362105 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0925 22:08:03.362110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:08:03.362123 (+ 13us) server_negotiation.cc:407] Connection header received
0925 22:08:03.362167 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:08:03.362169 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:08:03.362185 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:08:03.362263 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:08:03.362595 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.363725 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.365766 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.367003 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.367089 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:03.367143 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 22:08:03.367275 (+ 132us) server_negotiation.cc:299] Negotiation successful
0925 22:08:03.367315 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":72,"threads_started":1}
I20260925 22:08:03.368007 20764 tablet_service.cc:1467] Processing CreateTablet for tablet 5867217ab10c4eaa9059a741f8866553 (DEFAULT_TABLE table=test-table [id=f3beaa5d866a4f4e9e28ee4b6be0dcb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:08:03.368366 20764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5867217ab10c4eaa9059a741f8866553. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:08:03.370920 20981 tablet_bootstrap.cc:492] T 5867217ab10c4eaa9059a741f8866553 P 370bc5fa7b7445ac9d0d063337dc9587: Bootstrap starting.
I20260925 22:08:03.371068 20982 tablet_bootstrap.cc:492] T 5867217ab10c4eaa9059a741f8866553 P 38b225e2ae0f4f8e971038cc071def22: Bootstrap starting.
I20260925 22:08:03.371598 20982 tablet_bootstrap.cc:654] T 5867217ab10c4eaa9059a741f8866553 P 38b225e2ae0f4f8e971038cc071def22: Neither blocks nor log segments found. Creating new log.
I20260925 22:08:03.372112 20982 log.cc:824] T 5867217ab10c4eaa9059a741f8866553 P 38b225e2ae0f4f8e971038cc071def22: Log is configured to *not* fsync() on all Append() calls
I20260925 22:08:03.372419 20983 tablet_bootstrap.cc:492] T 5867217ab10c4eaa9059a741f8866553 P 69e6a6a4c260409d945be3ac9784d5e8: Bootstrap starting.
I20260925 22:08:03.372496 20981 tablet_bootstrap.cc:654] T 5867217ab10c4eaa9059a741f8866553 P 370bc5fa7b7445ac9d0d063337dc9587: Neither blocks nor log segments found. Creating new log.
I20260925 22:08:03.372812 20982 tablet_bootstrap.cc:492] T 5867217ab10c4eaa9059a741f8866553 P 38b225e2ae0f4f8e971038cc071def22: No bootstrap required, opened a new log
I20260925 22:08:03.372864 20982 ts_tablet_manager.cc:1397] T 5867217ab10c4eaa9059a741f8866553 P 38b225e2ae0f4f8e971038cc071def22: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:08:03.373039 20981 log.cc:824] T 5867217ab10c4eaa9059a741f8866553 P 370bc5fa7b7445ac9d0d063337dc9587: Log is configured to *not* fsync() on all Append() calls
I20260925 22:08:03.373701 20981 tablet_bootstrap.cc:492] T 5867217ab10c4eaa9059a741f8866553 P 370bc5fa7b7445ac9d0d063337dc9587: No bootstrap required, opened a new log
I20260925 22:08:03.373780 20981 ts_tablet_manager.cc:1397] T 5867217ab10c4eaa9059a741f8866553 P 370bc5fa7b7445ac9d0d063337dc9587: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260925 22:08:03.374552 20982 raft_consensus.cc:348] T 5867217ab10c4eaa9059a741f8866553 P 38b225e2ae0f4f8e971038cc071def22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e6a6a4c260409d945be3ac9784d5e8" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 41847 } } peers { permanent_uuid: "370bc5fa7b7445ac9d0d063337dc9587" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 39851 } } peers { permanent_uuid: "38b225e2ae0f4f8e971038cc071def22" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37951 } }
I20260925 22:08:03.374708 20982 raft_consensus.cc:374] T 5867217ab10c4eaa9059a741f8866553 P 38b225e2ae0f4f8e971038cc071def22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:08:03.374732 20982 raft_consensus.cc:729] T 5867217ab10c4eaa9059a741f8866553 P 38b225e2ae0f4f8e971038cc071def22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b225e2ae0f4f8e971038cc071def22, State: Initialized, Role: FOLLOWER
I20260925 22:08:03.374855 20982 consensus_queue.cc:260] T 5867217ab10c4eaa9059a741f8866553 P 38b225e2ae0f4f8e971038cc071def22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e6a6a4c260409d945be3ac9784d5e8" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 41847 } } peers { permanent_uuid: "370bc5fa7b7445ac9d0d063337dc9587" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 39851 } } peers { permanent_uuid: "38b225e2ae0f4f8e971038cc071def22" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37951 } }
I20260925 22:08:03.375128 20982 ts_tablet_manager.cc:1428] T 5867217ab10c4eaa9059a741f8866553 P 38b225e2ae0f4f8e971038cc071def22: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:08:03.375154 20696 heartbeater.cc:503] Master 127.5.0.126:42695 was elected leader, sending a full tablet report...
I20260925 22:08:03.375758 20983 tablet_bootstrap.cc:654] T 5867217ab10c4eaa9059a741f8866553 P 69e6a6a4c260409d945be3ac9784d5e8: Neither blocks nor log segments found. Creating new log.
I20260925 22:08:03.375869 20981 raft_consensus.cc:348] T 5867217ab10c4eaa9059a741f8866553 P 370bc5fa7b7445ac9d0d063337dc9587 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e6a6a4c260409d945be3ac9784d5e8" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 41847 } } peers { permanent_uuid: "370bc5fa7b7445ac9d0d063337dc9587" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 39851 } } peers { permanent_uuid: "38b225e2ae0f4f8e971038cc071def22" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37951 } }
I20260925 22:08:03.376216 20981 raft_consensus.cc:374] T 5867217ab10c4eaa9059a741f8866553 P 370bc5fa7b7445ac9d0d063337dc9587 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:08:03.376272 20981 raft_consensus.cc:729] T 5867217ab10c4eaa9059a741f8866553 P 370bc5fa7b7445ac9d0d063337dc9587 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 370bc5fa7b7445ac9d0d063337dc9587, State: Initialized, Role: FOLLOWER
I20260925 22:08:03.376386 20983 log.cc:824] T 5867217ab10c4eaa9059a741f8866553 P 69e6a6a4c260409d945be3ac9784d5e8: Log is configured to *not* fsync() on all Append() calls
I20260925 22:08:03.376431 20981 consensus_queue.cc:260] T 5867217ab10c4eaa9059a741f8866553 P 370bc5fa7b7445ac9d0d063337dc9587 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e6a6a4c260409d945be3ac9784d5e8" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 41847 } } peers { permanent_uuid: "370bc5fa7b7445ac9d0d063337dc9587" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 39851 } } peers { permanent_uuid: "38b225e2ae0f4f8e971038cc071def22" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37951 } }
I20260925 22:08:03.376713 20964 heartbeater.cc:503] Master 127.5.0.126:42695 was elected leader, sending a full tablet report...
I20260925 22:08:03.376951 20981 ts_tablet_manager.cc:1428] T 5867217ab10c4eaa9059a741f8866553 P 370bc5fa7b7445ac9d0d063337dc9587: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 22:08:03.377224 20983 tablet_bootstrap.cc:492] T 5867217ab10c4eaa9059a741f8866553 P 69e6a6a4c260409d945be3ac9784d5e8: No bootstrap required, opened a new log
I20260925 22:08:03.377313 20983 ts_tablet_manager.cc:1397] T 5867217ab10c4eaa9059a741f8866553 P 69e6a6a4c260409d945be3ac9784d5e8: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260925 22:08:03.378898 20983 raft_consensus.cc:348] T 5867217ab10c4eaa9059a741f8866553 P 69e6a6a4c260409d945be3ac9784d5e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e6a6a4c260409d945be3ac9784d5e8" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 41847 } } peers { permanent_uuid: "370bc5fa7b7445ac9d0d063337dc9587" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 39851 } } peers { permanent_uuid: "38b225e2ae0f4f8e971038cc071def22" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37951 } }
I20260925 22:08:03.379019 20983 raft_consensus.cc:374] T 5867217ab10c4eaa9059a741f8866553 P 69e6a6a4c260409d945be3ac9784d5e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:08:03.379040 20983 raft_consensus.cc:729] T 5867217ab10c4eaa9059a741f8866553 P 69e6a6a4c260409d945be3ac9784d5e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69e6a6a4c260409d945be3ac9784d5e8, State: Initialized, Role: FOLLOWER
I20260925 22:08:03.379119 20983 consensus_queue.cc:260] T 5867217ab10c4eaa9059a741f8866553 P 69e6a6a4c260409d945be3ac9784d5e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e6a6a4c260409d945be3ac9784d5e8" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 41847 } } peers { permanent_uuid: "370bc5fa7b7445ac9d0d063337dc9587" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 39851 } } peers { permanent_uuid: "38b225e2ae0f4f8e971038cc071def22" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37951 } }
I20260925 22:08:03.379299 20983 ts_tablet_manager.cc:1428] T 5867217ab10c4eaa9059a741f8866553 P 69e6a6a4c260409d945be3ac9784d5e8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:08:03.379367 20830 heartbeater.cc:503] Master 127.5.0.126:42695 was elected leader, sending a full tablet report...
W20260925 22:08:03.421623 20831 tablet.cc:2369] T 5867217ab10c4eaa9059a741f8866553 P 69e6a6a4c260409d945be3ac9784d5e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 22:08:03.517630 20697 tablet.cc:2369] T 5867217ab10c4eaa9059a741f8866553 P 38b225e2ae0f4f8e971038cc071def22: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:08:03.544246 20988 raft_consensus.cc:482] T 5867217ab10c4eaa9059a741f8866553 P 370bc5fa7b7445ac9d0d063337dc9587 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:08:03.544373 20988 raft_consensus.cc:504] T 5867217ab10c4eaa9059a741f8866553 P 370bc5fa7b7445ac9d0d063337dc9587 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e6a6a4c260409d945be3ac9784d5e8" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 41847 } } peers { permanent_uuid: "370bc5fa7b7445ac9d0d063337dc9587" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 39851 } } peers { permanent_uuid: "38b225e2ae0f4f8e971038cc071def22" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37951 } }
I20260925 22:08:03.544761 20988 leader_election.cc:288] T 5867217ab10c4eaa9059a741f8866553 P 370bc5fa7b7445ac9d0d063337dc9587 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 69e6a6a4c260409d945be3ac9784d5e8 (127.5.0.66:41847), 38b225e2ae0f4f8e971038cc071def22 (127.5.0.65:37951)
I20260925 22:08:03.548820 20980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:03.545210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:35251 (local address 127.5.0.65:37951)
0925 22:08:03.545257 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 22:08:03.545259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:08:03.545375 (+ 116us) server_negotiation.cc:407] Connection header received
0925 22:08:03.545405 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:08:03.545406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:08:03.545418 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:08:03.545476 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:08:03.545625 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.546333 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.547868 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.548447 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.548535 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:03.548545 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 22:08:03.548701 (+ 156us) server_negotiation.cc:299] Negotiation successful
0925 22:08:03.548742 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 22:08:03.548820 20978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:03.545056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:46711 (local address 127.5.0.66:41847)
0925 22:08:03.545120 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 22:08:03.545122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:08:03.545155 (+ 33us) server_negotiation.cc:407] Connection header received
0925 22:08:03.545261 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:08:03.545263 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:08:03.545278 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:08:03.545367 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:08:03.545568 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.546338 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.547868 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.548459 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.548528 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:03.548539 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0925 22:08:03.548702 (+ 163us) server_negotiation.cc:299] Negotiation successful
0925 22:08:03.548744 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 22:08:03.548835 20966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:03.544980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:41847 (local address 127.5.0.67:46711)
0925 22:08:03.545093 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:03.545108 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 22:08:03.545174 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:08:03.545372 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:08:03.545373 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:08:03.545375 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:08:03.545480 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:03.545483 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.546429 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.546430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:03.547756 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:03.547759 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.548537 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.548539 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:03.548621 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:03.548623 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 22:08:03.548684 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 22:08:03.548734 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":38}
I20260925 22:08:03.548842 20990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:03.545134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:37951 (local address 127.5.0.67:35251)
0925 22:08:03.545298 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:03.545307 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:08:03.545352 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:08:03.545491 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:08:03.545492 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:08:03.545493 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:08:03.545553 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:03.545555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.546429 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.546430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:03.547756 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:03.547759 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.548538 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.548539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:03.548619 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:03.548623 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:08:03.548684 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 22:08:03.548734 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":103,"thread_start_us":112,"threads_started":1}
I20260925 22:08:03.549263 20784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5867217ab10c4eaa9059a741f8866553" candidate_uuid: "370bc5fa7b7445ac9d0d063337dc9587" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69e6a6a4c260409d945be3ac9784d5e8" is_pre_election: true
I20260925 22:08:03.549350 20650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5867217ab10c4eaa9059a741f8866553" candidate_uuid: "370bc5fa7b7445ac9d0d063337dc9587" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b225e2ae0f4f8e971038cc071def22" is_pre_election: true
I20260925 22:08:03.549443 20784 raft_consensus.cc:2454] T 5867217ab10c4eaa9059a741f8866553 P 69e6a6a4c260409d945be3ac9784d5e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 370bc5fa7b7445ac9d0d063337dc9587 in term 0.
I20260925 22:08:03.549522 20650 raft_consensus.cc:2454] T 5867217ab10c4eaa9059a741f8866553 P 38b225e2ae0f4f8e971038cc071def22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 370bc5fa7b7445ac9d0d063337dc9587 in term 0.
I20260925 22:08:03.549700 20854 leader_election.cc:302] T 5867217ab10c4eaa9059a741f8866553 P 370bc5fa7b7445ac9d0d063337dc9587 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 370bc5fa7b7445ac9d0d063337dc9587, 69e6a6a4c260409d945be3ac9784d5e8; no voters:
I20260925 22:08:03.549876 20988 raft_consensus.cc:2781] T 5867217ab10c4eaa9059a741f8866553 P 370bc5fa7b7445ac9d0d063337dc9587 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:08:03.549957 20988 raft_consensus.cc:482] T 5867217ab10c4eaa9059a741f8866553 P 370bc5fa7b7445ac9d0d063337dc9587 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:08:03.550001 20988 raft_consensus.cc:3037] T 5867217ab10c4eaa9059a741f8866553 P 370bc5fa7b7445ac9d0d063337dc9587 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:08:03.550747 20988 raft_consensus.cc:504] T 5867217ab10c4eaa9059a741f8866553 P 370bc5fa7b7445ac9d0d063337dc9587 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e6a6a4c260409d945be3ac9784d5e8" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 41847 } } peers { permanent_uuid: "370bc5fa7b7445ac9d0d063337dc9587" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 39851 } } peers { permanent_uuid: "38b225e2ae0f4f8e971038cc071def22" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37951 } }
I20260925 22:08:03.550907 20988 leader_election.cc:288] T 5867217ab10c4eaa9059a741f8866553 P 370bc5fa7b7445ac9d0d063337dc9587 [CANDIDATE]: Term 1 election: Requested vote from peers 69e6a6a4c260409d945be3ac9784d5e8 (127.5.0.66:41847), 38b225e2ae0f4f8e971038cc071def22 (127.5.0.65:37951)
I20260925 22:08:03.551098 20784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5867217ab10c4eaa9059a741f8866553" candidate_uuid: "370bc5fa7b7445ac9d0d063337dc9587" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69e6a6a4c260409d945be3ac9784d5e8"
I20260925 22:08:03.551095 20650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5867217ab10c4eaa9059a741f8866553" candidate_uuid: "370bc5fa7b7445ac9d0d063337dc9587" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b225e2ae0f4f8e971038cc071def22"
I20260925 22:08:03.551160 20784 raft_consensus.cc:3037] T 5867217ab10c4eaa9059a741f8866553 P 69e6a6a4c260409d945be3ac9784d5e8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:08:03.551182 20650 raft_consensus.cc:3037] T 5867217ab10c4eaa9059a741f8866553 P 38b225e2ae0f4f8e971038cc071def22 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:08:03.551841 20784 raft_consensus.cc:2454] T 5867217ab10c4eaa9059a741f8866553 P 69e6a6a4c260409d945be3ac9784d5e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 370bc5fa7b7445ac9d0d063337dc9587 in term 1.
I20260925 22:08:03.551856 20650 raft_consensus.cc:2454] T 5867217ab10c4eaa9059a741f8866553 P 38b225e2ae0f4f8e971038cc071def22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 370bc5fa7b7445ac9d0d063337dc9587 in term 1.
I20260925 22:08:03.552027 20854 leader_election.cc:302] T 5867217ab10c4eaa9059a741f8866553 P 370bc5fa7b7445ac9d0d063337dc9587 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 370bc5fa7b7445ac9d0d063337dc9587, 38b225e2ae0f4f8e971038cc071def22; no voters:
I20260925 22:08:03.552127 20988 raft_consensus.cc:2781] T 5867217ab10c4eaa9059a741f8866553 P 370bc5fa7b7445ac9d0d063337dc9587 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:08:03.552287 20988 raft_consensus.cc:686] T 5867217ab10c4eaa9059a741f8866553 P 370bc5fa7b7445ac9d0d063337dc9587 [term 1 LEADER]: Becoming Leader. State: Replica: 370bc5fa7b7445ac9d0d063337dc9587, State: Running, Role: LEADER
I20260925 22:08:03.552384 20988 consensus_queue.cc:237] T 5867217ab10c4eaa9059a741f8866553 P 370bc5fa7b7445ac9d0d063337dc9587 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e6a6a4c260409d945be3ac9784d5e8" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 41847 } } peers { permanent_uuid: "370bc5fa7b7445ac9d0d063337dc9587" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 39851 } } peers { permanent_uuid: "38b225e2ae0f4f8e971038cc071def22" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37951 } }
I20260925 22:08:03.553274 20505 catalog_manager.cc:5495] T 5867217ab10c4eaa9059a741f8866553 P 370bc5fa7b7445ac9d0d063337dc9587 reported cstate change: term changed from 0 to 1, leader changed from <none> to 370bc5fa7b7445ac9d0d063337dc9587 (127.5.0.67). New cstate: current_term: 1 leader_uuid: "370bc5fa7b7445ac9d0d063337dc9587" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e6a6a4c260409d945be3ac9784d5e8" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 41847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "370bc5fa7b7445ac9d0d063337dc9587" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 39851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38b225e2ae0f4f8e971038cc071def22" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37951 } health_report { overall_health: UNKNOWN } } }
W20260925 22:08:03.583253 20965 tablet.cc:2369] T 5867217ab10c4eaa9059a741f8866553 P 370bc5fa7b7445ac9d0d063337dc9587: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:08:03.592731 20977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:03.590502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44748 (local address 127.5.0.67:39851)
0925 22:08:03.590566 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 22:08:03.590568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:08:03.590578 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:08:03.590624 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:08:03.590625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:08:03.590643 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:08:03.590696 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:08:03.590819 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.591294 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.592031 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:03.592222 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:03.592289 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:03.592414 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:08:03.592512 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:08:03.592575 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 22:08:03.592595 (+ 20us) server_negotiation.cc:299] Negotiation successful
0925 22:08:03.592614 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 22:08:03.594502 20784 raft_consensus.cc:1261] T 5867217ab10c4eaa9059a741f8866553 P 69e6a6a4c260409d945be3ac9784d5e8 [term 1 FOLLOWER]: Refusing update from remote peer 370bc5fa7b7445ac9d0d063337dc9587: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:08:03.594549 20650 raft_consensus.cc:1261] T 5867217ab10c4eaa9059a741f8866553 P 38b225e2ae0f4f8e971038cc071def22 [term 1 FOLLOWER]: Refusing update from remote peer 370bc5fa7b7445ac9d0d063337dc9587: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:08:03.594897 20991 consensus_queue.cc:1035] T 5867217ab10c4eaa9059a741f8866553 P 370bc5fa7b7445ac9d0d063337dc9587 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b225e2ae0f4f8e971038cc071def22" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:08:03.595028 20988 consensus_queue.cc:1035] T 5867217ab10c4eaa9059a741f8866553 P 370bc5fa7b7445ac9d0d063337dc9587 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69e6a6a4c260409d945be3ac9784d5e8" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 41847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:08:03.597568 20996 mvcc.cc:205] Tried to move back new op lower bound from 7333372246400380928 to 7333372246231203840. Current Snapshot: MvccSnapshot[applied={T|T < 7333372246400380928}]
I20260925 22:08:03.597630 20994 mvcc.cc:205] Tried to move back new op lower bound from 7333372246400380928 to 7333372246231203840. Current Snapshot: MvccSnapshot[applied={T|T < 7333372246400380928}]
I20260925 22:08:03.600427 20504 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40506:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:08:03.600536 20504 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40506:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:08:03.601277 20504 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d: Sending DeleteTablet for 3 replicas of tablet 5867217ab10c4eaa9059a741f8866553
I20260925 22:08:03.601620 20764 tablet_service.cc:1514] Processing DeleteTablet for tablet 5867217ab10c4eaa9059a741f8866553 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:08:03 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:33688
I20260925 22:08:03.601622 20630 tablet_service.cc:1514] Processing DeleteTablet for tablet 5867217ab10c4eaa9059a741f8866553 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:08:03 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:43134
I20260925 22:08:03.601817 20898 tablet_service.cc:1514] Processing DeleteTablet for tablet 5867217ab10c4eaa9059a741f8866553 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:08:03 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:44744
I20260925 22:08:03.601909 21005 tablet_replica.cc:330] T 5867217ab10c4eaa9059a741f8866553 P 69e6a6a4c260409d945be3ac9784d5e8: stopping tablet replica
I20260925 22:08:03.601909 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 20567
I20260925 22:08:03.602003 21005 raft_consensus.cc:2229] T 5867217ab10c4eaa9059a741f8866553 P 69e6a6a4c260409d945be3ac9784d5e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:08:03.602169 21005 raft_consensus.cc:2258] T 5867217ab10c4eaa9059a741f8866553 P 69e6a6a4c260409d945be3ac9784d5e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:08:03.602505 21006 tablet_replica.cc:330] T 5867217ab10c4eaa9059a741f8866553 P 370bc5fa7b7445ac9d0d063337dc9587: stopping tablet replica
I20260925 22:08:03.602604 21006 raft_consensus.cc:2229] T 5867217ab10c4eaa9059a741f8866553 P 370bc5fa7b7445ac9d0d063337dc9587 [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:08:03.602723 21006 raft_consensus.cc:2258] T 5867217ab10c4eaa9059a741f8866553 P 370bc5fa7b7445ac9d0d063337dc9587 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:08:03.603130 21006 ts_tablet_manager.cc:1905] T 5867217ab10c4eaa9059a741f8866553 P 370bc5fa7b7445ac9d0d063337dc9587: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:08:03.603801 21005 ts_tablet_manager.cc:1905] T 5867217ab10c4eaa9059a741f8866553 P 69e6a6a4c260409d945be3ac9784d5e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:08:03.605974 21005 ts_tablet_manager.cc:1918] T 5867217ab10c4eaa9059a741f8866553 P 69e6a6a4c260409d945be3ac9784d5e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:08:03.606048 21005 log.cc:1196] T 5867217ab10c4eaa9059a741f8866553 P 69e6a6a4c260409d945be3ac9784d5e8: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/wal/wals/5867217ab10c4eaa9059a741f8866553
I20260925 22:08:03.606316 21005 ts_tablet_manager.cc:1939] T 5867217ab10c4eaa9059a741f8866553 P 69e6a6a4c260409d945be3ac9784d5e8: Deleting consensus metadata
I20260925 22:08:03.606774 20492 catalog_manager.cc:4841] TS 69e6a6a4c260409d945be3ac9784d5e8 (127.5.0.66:41847): tablet 5867217ab10c4eaa9059a741f8866553 (table test-table [id=f3beaa5d866a4f4e9e28ee4b6be0dcb8]) successfully deleted
I20260925 22:08:03.608426 21006 ts_tablet_manager.cc:1918] T 5867217ab10c4eaa9059a741f8866553 P 370bc5fa7b7445ac9d0d063337dc9587: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:08:03.608510 21006 log.cc:1196] T 5867217ab10c4eaa9059a741f8866553 P 370bc5fa7b7445ac9d0d063337dc9587: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/wal/wals/5867217ab10c4eaa9059a741f8866553
I20260925 22:08:03.608768 21006 ts_tablet_manager.cc:1939] T 5867217ab10c4eaa9059a741f8866553 P 370bc5fa7b7445ac9d0d063337dc9587: Deleting consensus metadata
I20260925 22:08:03.609287 20489 catalog_manager.cc:4841] TS 370bc5fa7b7445ac9d0d063337dc9587 (127.5.0.67:39851): tablet 5867217ab10c4eaa9059a741f8866553 (table test-table [id=f3beaa5d866a4f4e9e28ee4b6be0dcb8]) successfully deleted
W20260925 22:08:03.609562 20492 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.0.65:37951 (error 108)
I20260925 22:08:03.609733 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 20701
I20260925 22:08:03.612756 20976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:03.612608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:37951 (local address 127.0.0.1:43136)
0925 22:08:03.612675 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:03.612720 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.65:37951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260925 22:08:03.612887 20492 catalog_manager.cc:4584] TS 38b225e2ae0f4f8e971038cc071def22 (127.5.0.65:37951): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5867217ab10c4eaa9059a741f8866553: Network error: Client connection negotiation failed: client connection to 127.5.0.65:37951: connect: Connection refused (error 111)
I20260925 22:08:03.614337 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 20835
I20260925 22:08:03.619267 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 20473
I20260925 22:08:23.624727 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:42695
--webserver_interface=127.5.0.126
--webserver_port=37595
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:37557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:42695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:08:23.705639 21008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:08:23.705849 21008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:08:23.705873 21008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:08:23.706979 21008 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 22:08:23.707012 21008 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 22:08:23.707026 21008 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 22:08:23.707037 21008 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:08:23.707049 21008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:08:23.707062 21008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:08:23.707074 21008 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:08:23.707086 21008 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:08:23.708487 21008 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:37557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:42695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:42695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=37595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:08:23.708685 21008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:08:23.708853 21008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:08:23.717512 21014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:08:23.717512 21015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:08:23.717558 21017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:08:23.717767 21008 server_base.cc:1034] running on GCE node
I20260925 22:08:23.718096 21008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:08:23.718405 21008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:08:23.719579 21008 hybrid_clock.cc:648] HybridClock initialized: now 1790374103719568 us; error 32 us; skew 500 ppm
Sep 25 22:08:23 dist-test-slave-7mqr krb5kdc[20460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374103, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:08:23.722318 21008 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:08:23.722782 21008 webserver.cc:460] Webserver started at http://127.5.0.126:37595/ using document root <none> and password file <none>
I20260925 22:08:23.722992 21008 fs_manager.cc:359] Metadata directory not provided
I20260925 22:08:23.723045 21008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:08:23.724611 21008 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:08:23.725348 21024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:08:23.725611 21008 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:08:23.725719 21008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "521bc070528545ebb74dce96dfc41a2d"
format_stamp: "Formatted at 2026-09-25 22:08:02 on dist-test-slave-7mqr"
server_key: "0ad726727fd5c79765dbef07dc511f23"
server_key_iv: "166fd940d02c1fc69bd05205333f223b"
server_key_version: "encryptionkey@0"
I20260925 22:08:23.726197 21008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:08:23.737758 21008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:08:23.739787 21008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:08:23.739998 21008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:08:23.744433 21008 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:42695
I20260925 22:08:23.744514 21076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:42695 every 8 connection(s)
I20260925 22:08:23.744925 21008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:08:23.746419 21077 sys_catalog.cc:263] Verifying existing consensus state
I20260925 22:08:23.747126 21077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d: Bootstrap starting.
I20260925 22:08:23.749891 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 21008
I20260925 22:08:23.750232 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.65:37951
--local_ip_for_outbound_sockets=127.5.0.65
--tserver_master_addrs=127.5.0.126:42695
--webserver_port=42041
--webserver_interface=127.5.0.65
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:37557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:08:23.755738 21077 log.cc:824] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d: Log is configured to *not* fsync() on all Append() calls
I20260925 22:08:23.757642 21077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 22:08:23.757929 21077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d: Bootstrap complete.
I20260925 22:08:23.760278 21077 raft_consensus.cc:348] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521bc070528545ebb74dce96dfc41a2d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42695 } }
I20260925 22:08:23.760532 21077 raft_consensus.cc:729] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 521bc070528545ebb74dce96dfc41a2d, State: Initialized, Role: FOLLOWER
I20260925 22:08:23.760721 21077 consensus_queue.cc:260] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521bc070528545ebb74dce96dfc41a2d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42695 } }
I20260925 22:08:23.760800 21077 raft_consensus.cc:388] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:08:23.760859 21077 raft_consensus.cc:482] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:08:23.760912 21077 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [term 1 FOLLOWER]: Advancing to term 2
I20260925 22:08:23.761891 21077 raft_consensus.cc:504] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521bc070528545ebb74dce96dfc41a2d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42695 } }
I20260925 22:08:23.762015 21077 leader_election.cc:302] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 521bc070528545ebb74dce96dfc41a2d; no voters:
I20260925 22:08:23.762216 21077 leader_election.cc:288] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 22:08:23.762251 21081 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [term 2 FOLLOWER]: Leader election won for term 2
I20260925 22:08:23.762385 21081 raft_consensus.cc:686] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [term 2 LEADER]: Becoming Leader. State: Replica: 521bc070528545ebb74dce96dfc41a2d, State: Running, Role: LEADER
I20260925 22:08:23.762495 21077 sys_catalog.cc:564] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:08:23.762562 21081 consensus_queue.cc:237] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521bc070528545ebb74dce96dfc41a2d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42695 } }
I20260925 22:08:23.762934 21082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "521bc070528545ebb74dce96dfc41a2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521bc070528545ebb74dce96dfc41a2d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42695 } } }
I20260925 22:08:23.762945 21083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 521bc070528545ebb74dce96dfc41a2d. Latest consensus state: current_term: 2 leader_uuid: "521bc070528545ebb74dce96dfc41a2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521bc070528545ebb74dce96dfc41a2d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42695 } } }
I20260925 22:08:23.763044 21082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [sys.catalog]: This master's current role is: LEADER
I20260925 22:08:23.763065 21083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [sys.catalog]: This master's current role is: LEADER
I20260925 22:08:23.763720 21089 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:08:23.764711 21089 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:08:23.764820 21089 catalog_manager.cc:1259] Loaded cluster ID: 92e572e87e274bee8b613af0ca1dfb9e
I20260925 22:08:23.764855 21089 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:08:23.766227 21089 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:08:23.766448 21089 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d: Loaded TSK: 0
I20260925 22:08:23.783617 21089 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d: Generated new TSK 1
I20260925 22:08:23.783743 21089 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 22:08:23.843149 21079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:08:23.843340 21079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:08:23.843361 21079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:08:23.844547 21079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:08:23.844585 21079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:08:23.844612 21079 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.65
I20260925 22:08:23.846038 21079 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:37557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.65:37951
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.0.65
--webserver_port=42041
--tserver_master_addrs=127.5.0.126:42695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21079
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.65
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:08:23.846287 21079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:08:23.846550 21079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:08:23.852617 21105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:08:23.852658 21107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:08:23.852864 21104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:08:23.853134 21079 server_base.cc:1034] running on GCE node
I20260925 22:08:23.853341 21079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:08:23.853590 21079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:08:23.854751 21079 hybrid_clock.cc:648] HybridClock initialized: now 1790374103854711 us; error 58 us; skew 500 ppm
Sep 25 22:08:23 dist-test-slave-7mqr krb5kdc[20460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374103, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:08:23.857492 21079 init.cc:377] Logged in from keytab as kudu/127.5.0.65@KRBTEST.COM (short username kudu)
I20260925 22:08:23.858016 21079 webserver.cc:460] Webserver started at http://127.5.0.65:42041/ using document root <none> and password file <none>
I20260925 22:08:23.858310 21079 fs_manager.cc:359] Metadata directory not provided
I20260925 22:08:23.858402 21079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:08:23.859974 21079 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 22:08:23.860730 21114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:08:23.860911 21079 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:08:23.861021 21079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/wal
uuid: "38b225e2ae0f4f8e971038cc071def22"
format_stamp: "Formatted at 2026-09-25 22:08:02 on dist-test-slave-7mqr"
server_key: "938f1b6a9bedd22127b0db309cfa085d"
server_key_iv: "94b871fada6e304e219c5560df27ead0"
server_key_version: "encryptionkey@0"
I20260925 22:08:23.861395 21079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:08:23.877823 21079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:08:23.880565 21079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:08:23.880767 21079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:08:23.881042 21079 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:08:23.881623 21121 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 22:08:23.883219 21079 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 22:08:23.883281 21079 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260925 22:08:23.883338 21079 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 22:08:23.884127 21079 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 22:08:23.884176 21079 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260925 22:08:23.884222 21121 tablet_bootstrap.cc:492] T 5867217ab10c4eaa9059a741f8866553 P 38b225e2ae0f4f8e971038cc071def22: Bootstrap starting.
I20260925 22:08:23.892773 21079 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.65:37951
I20260925 22:08:23.892809 21228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.65:37951 every 8 connection(s)
I20260925 22:08:23.893358 21079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
I20260925 22:08:23.896024 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 21079
I20260925 22:08:23.896356 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.66:41847
--local_ip_for_outbound_sockets=127.5.0.66
--tserver_master_addrs=127.5.0.126:42695
--webserver_port=45095
--webserver_interface=127.5.0.66
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:37557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:08:23.896401 21121 log.cc:824] T 5867217ab10c4eaa9059a741f8866553 P 38b225e2ae0f4f8e971038cc071def22: Log is configured to *not* fsync() on all Append() calls
I20260925 22:08:23.899165 21121 tablet_bootstrap.cc:492] T 5867217ab10c4eaa9059a741f8866553 P 38b225e2ae0f4f8e971038cc071def22: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 22:08:23.899528 21121 tablet_bootstrap.cc:492] T 5867217ab10c4eaa9059a741f8866553 P 38b225e2ae0f4f8e971038cc071def22: Bootstrap complete.
I20260925 22:08:23.900827 21121 ts_tablet_manager.cc:1397] T 5867217ab10c4eaa9059a741f8866553 P 38b225e2ae0f4f8e971038cc071def22: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.012s
Sep 25 22:08:23 dist-test-slave-7mqr krb5kdc[20460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374103, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:08:23.902778 21121 raft_consensus.cc:348] T 5867217ab10c4eaa9059a741f8866553 P 38b225e2ae0f4f8e971038cc071def22 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e6a6a4c260409d945be3ac9784d5e8" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 41847 } } peers { permanent_uuid: "370bc5fa7b7445ac9d0d063337dc9587" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 39851 } } peers { permanent_uuid: "38b225e2ae0f4f8e971038cc071def22" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37951 } }
I20260925 22:08:23.903211 21121 raft_consensus.cc:729] T 5867217ab10c4eaa9059a741f8866553 P 38b225e2ae0f4f8e971038cc071def22 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b225e2ae0f4f8e971038cc071def22, State: Initialized, Role: FOLLOWER
I20260925 22:08:23.903477 21121 consensus_queue.cc:260] T 5867217ab10c4eaa9059a741f8866553 P 38b225e2ae0f4f8e971038cc071def22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e6a6a4c260409d945be3ac9784d5e8" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 41847 } } peers { permanent_uuid: "370bc5fa7b7445ac9d0d063337dc9587" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 39851 } } peers { permanent_uuid: "38b225e2ae0f4f8e971038cc071def22" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37951 } }
I20260925 22:08:23.903928 21121 ts_tablet_manager.cc:1428] T 5867217ab10c4eaa9059a741f8866553 P 38b225e2ae0f4f8e971038cc071def22: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260925 22:08:23.906883 21231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:23.894240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.65:50447)
0925 22:08:23.896045 (+ 1805us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:23.896100 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 22:08:23.896862 (+ 762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:08:23.897418 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:08:23.897423 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:08:23.897827 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:08:23.898133 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:23.898140 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:23.899527 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:23.899529 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:23.900483 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:23.900486 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:23.901164 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:23.901166 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:23.901302 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:23.901869 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:08:23.901885 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:08:23.903282 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:08:23.905234 (+ 1952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:08:23.905241 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:08:23.905243 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:08:23.905479 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:08:23.905748 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:08:23.905749 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:08:23.905749 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:08:23.905859 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:08:23.906241 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:08:23.906248 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:08:23.906386 (+ 138us) client_negotiation.cc:769] Sending connection context
0925 22:08:23.906510 (+ 124us) client_negotiation.cc:240] Negotiation successful
0925 22:08:23.906685 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1750,"thread_start_us":96,"threads_started":1}
I20260925 22:08:23.906883 21232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:23.894383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:50447 (local address 127.5.0.126:42695)
0925 22:08:23.896723 (+ 2340us) server_negotiation.cc:206] Beginning negotiation
0925 22:08:23.896735 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 22:08:23.896789 (+ 54us) server_negotiation.cc:407] Connection header received
0925 22:08:23.897125 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:08:23.897134 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:08:23.897198 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:08:23.897357 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:08:23.898280 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:23.899398 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:23.900586 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:23.901048 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:23.901160 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:23.903438 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:08:23.903442 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:08:23.903442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:08:23.903453 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:08:23.905128 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:08:23.905570 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:08:23.905573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:08:23.905574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:08:23.905674 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:08:23.905930 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:08:23.905931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:08:23.905931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:08:23.906155 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:08:23.906248 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 22:08:23.906566 (+ 318us) server_negotiation.cc:299] Negotiation successful
0925 22:08:23.906694 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2277,"thread_start_us":80,"threads_started":1}
I20260925 22:08:23.907532 21229 heartbeater.cc:347] Connected to a master server at 127.5.0.126:42695
I20260925 22:08:23.907647 21229 heartbeater.cc:464] Registering TS with master...
I20260925 22:08:23.907810 21229 heartbeater.cc:511] Master 127.5.0.126:42695 requested a full tablet report, sending...
I20260925 22:08:23.908553 21041 ts_manager.cc:194] Registered new tserver with Master: 38b225e2ae0f4f8e971038cc071def22 (127.5.0.65:37951)
I20260925 22:08:23.909598 21041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.65@KRBTEST.COM'} at 127.5.0.65:50447
I20260925 22:08:23.910509 21229 heartbeater.cc:503] Master 127.5.0.126:42695 was elected leader, sending a full tablet report...
Sep 25 22:08:23 dist-test-slave-7mqr krb5kdc[20460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374103, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for kudu/127.5.0.65@KRBTEST.COM
I20260925 22:08:23.918155 21236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:23.909162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:37951 (local address 127.0.0.1:41558)
0925 22:08:23.909352 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:23.909392 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 22:08:23.909455 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:08:23.909677 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:08:23.909682 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:08:23.910009 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:08:23.910055 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:23.910057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:23.911458 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:23.911459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:23.912110 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:23.912112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:23.912798 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:23.912801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:23.912860 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:23.913378 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:08:23.913389 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:08:23.914737 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:08:23.917032 (+ 2295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:08:23.917038 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:08:23.917039 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:08:23.917237 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:08:23.917461 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:08:23.917462 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:08:23.917463 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:08:23.917520 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:08:23.917844 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:08:23.917851 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:08:23.917899 (+ 48us) client_negotiation.cc:769] Sending connection context
0925 22:08:23.917986 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 22:08:23.918081 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":115,"threads_started":1}
I20260925 22:08:23.918237 21237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:23.909235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41558 (local address 127.5.0.65:37951)
0925 22:08:23.909418 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0925 22:08:23.909423 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:08:23.909439 (+ 16us) server_negotiation.cc:407] Connection header received
0925 22:08:23.909562 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:08:23.909564 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:08:23.909579 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:08:23.909666 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:08:23.910141 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:23.911358 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:23.912185 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:23.912713 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:23.912795 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:23.914820 (+ 2025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:08:23.914827 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:08:23.914827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:08:23.914834 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:08:23.916921 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:08:23.917321 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:08:23.917324 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:08:23.917325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:08:23.917389 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:08:23.917583 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:08:23.917584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:08:23.917584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:08:23.917755 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:08:23.917835 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 22:08:23.918042 (+ 207us) server_negotiation.cc:299] Negotiation successful
0925 22:08:23.918140 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":68,"threads_started":1}
I20260925 22:08:23.918713 21163 tablet_service.cc:1514] Processing DeleteTablet for tablet 5867217ab10c4eaa9059a741f8866553 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:08:03 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:41558
I20260925 22:08:23.919255 21238 tablet_replica.cc:330] stopping tablet replica
I20260925 22:08:23.919378 21238 raft_consensus.cc:2229] T 5867217ab10c4eaa9059a741f8866553 P 38b225e2ae0f4f8e971038cc071def22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:08:23.919523 21238 raft_consensus.cc:2258] T 5867217ab10c4eaa9059a741f8866553 P 38b225e2ae0f4f8e971038cc071def22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:08:23.919842 21238 ts_tablet_manager.cc:1905] T 5867217ab10c4eaa9059a741f8866553 P 38b225e2ae0f4f8e971038cc071def22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:08:23.922000 21238 ts_tablet_manager.cc:1918] T 5867217ab10c4eaa9059a741f8866553 P 38b225e2ae0f4f8e971038cc071def22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:08:23.922065 21238 log.cc:1196] T 5867217ab10c4eaa9059a741f8866553 P 38b225e2ae0f4f8e971038cc071def22: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/wal/wals/5867217ab10c4eaa9059a741f8866553
I20260925 22:08:23.922305 21238 ts_tablet_manager.cc:1939] T 5867217ab10c4eaa9059a741f8866553 P 38b225e2ae0f4f8e971038cc071def22: Deleting consensus metadata
I20260925 22:08:23.922857 21028 catalog_manager.cc:4841] TS 38b225e2ae0f4f8e971038cc071def22 (127.5.0.65:37951): tablet 5867217ab10c4eaa9059a741f8866553 (table test-table [id=f3beaa5d866a4f4e9e28ee4b6be0dcb8]) successfully deleted
W20260925 22:08:23.985323 21234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:08:23.985567 21234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:08:23.985605 21234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:08:23.986841 21234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:08:23.986905 21234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:08:23.986951 21234 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.66
I20260925 22:08:23.988416 21234 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:37557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.66:41847
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.0.66
--webserver_port=45095
--tserver_master_addrs=127.5.0.126:42695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.66
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:08:23.988674 21234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:08:23.988900 21234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:08:23.993871 21246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:08:23.993935 21244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:08:23.993968 21234 server_base.cc:1034] running on GCE node
W20260925 22:08:23.993875 21243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:08:23.994395 21234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:08:23.994637 21234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:08:23.995771 21234 hybrid_clock.cc:648] HybridClock initialized: now 1790374103995761 us; error 28 us; skew 500 ppm
Sep 25 22:08:23 dist-test-slave-7mqr krb5kdc[20460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374103, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:08:23.998147 21234 init.cc:377] Logged in from keytab as kudu/127.5.0.66@KRBTEST.COM (short username kudu)
I20260925 22:08:23.998591 21234 webserver.cc:460] Webserver started at http://127.5.0.66:45095/ using document root <none> and password file <none>
I20260925 22:08:23.998809 21234 fs_manager.cc:359] Metadata directory not provided
I20260925 22:08:23.998886 21234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:08:24.000207 21234 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:08:24.000803 21253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:08:24.000954 21234 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:08:24.001053 21234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/wal
uuid: "69e6a6a4c260409d945be3ac9784d5e8"
format_stamp: "Formatted at 2026-09-25 22:08:03 on dist-test-slave-7mqr"
server_key: "a0a60a847f3afade7b4469af792b3ab1"
server_key_iv: "90cf14e5e90db0adf0eba136598352d6"
server_key_version: "encryptionkey@0"
I20260925 22:08:24.001384 21234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:08:24.017568 21234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:08:24.019533 21234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:08:24.019691 21234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:08:24.020016 21234 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:08:24.020291 21234 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:08:24.020323 21234 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:08:24.020345 21234 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:08:24.020358 21234 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:08:24.026208 21234 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.66:41847
I20260925 22:08:24.026294 21366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.66:41847 every 8 connection(s)
I20260925 22:08:24.026652 21234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
I20260925 22:08:24.032936 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 21234
Sep 25 22:08:24 dist-test-slave-7mqr krb5kdc[20460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374103, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:08:24.033284 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.67:39851
--local_ip_for_outbound_sockets=127.5.0.67
--tserver_master_addrs=127.5.0.126:42695
--webserver_port=42047
--webserver_interface=127.5.0.67
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:37557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:08:24.037609 21232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:24.027428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:52171 (local address 127.5.0.126:42695)
0925 22:08:24.027518 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0925 22:08:24.027519 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:08:24.027731 (+ 212us) server_negotiation.cc:407] Connection header received
0925 22:08:24.028335 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:08:24.028336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:08:24.028352 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:08:24.028409 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:08:24.029364 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:24.029874 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:24.030886 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:24.031103 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:24.031190 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:24.033682 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:08:24.033683 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:08:24.033684 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:08:24.033690 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:08:24.035786 (+ 2096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:08:24.036336 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:08:24.036338 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:08:24.036338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:08:24.036431 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:08:24.036736 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:08:24.036737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:08:24.036737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:08:24.036923 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:08:24.036988 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 22:08:24.037351 (+ 363us) server_negotiation.cc:299] Negotiation successful
0925 22:08:24.037461 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 22:08:24.037688 21369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:24.027416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.66:52171)
0925 22:08:24.027607 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:24.027643 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 22:08:24.028225 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:08:24.028478 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:08:24.028482 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:08:24.029064 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:08:24.029253 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:24.029258 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:24.029980 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:24.029981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:24.030780 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:24.030782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:24.031200 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:24.031201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:24.031331 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:24.031786 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:08:24.031794 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:08:24.033533 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:08:24.035879 (+ 2346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:08:24.035912 (+ 33us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:08:24.035914 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:08:24.036255 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:08:24.036485 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:08:24.036486 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:08:24.036486 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:08:24.036651 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:08:24.036980 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:08:24.037027 (+ 47us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:08:24.037188 (+ 161us) client_negotiation.cc:769] Sending connection context
0925 22:08:24.037293 (+ 105us) client_negotiation.cc:240] Negotiation successful
0925 22:08:24.037462 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":92,"threads_started":1}
I20260925 22:08:24.038636 21367 heartbeater.cc:347] Connected to a master server at 127.5.0.126:42695
I20260925 22:08:24.038730 21367 heartbeater.cc:464] Registering TS with master...
I20260925 22:08:24.038909 21367 heartbeater.cc:511] Master 127.5.0.126:42695 requested a full tablet report, sending...
I20260925 22:08:24.039317 21041 ts_manager.cc:194] Registered new tserver with Master: 69e6a6a4c260409d945be3ac9784d5e8 (127.5.0.66:41847)
I20260925 22:08:24.039933 21041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.66@KRBTEST.COM'} at 127.5.0.66:52171
W20260925 22:08:24.117790 21370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:08:24.117987 21370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:08:24.118010 21370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:08:24.119148 21370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:08:24.119189 21370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:08:24.119241 21370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.67
I20260925 22:08:24.120760 21370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:37557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.67:39851
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.0.67
--webserver_port=42047
--tserver_master_addrs=127.5.0.126:42695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.67
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:08:24.120997 21370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:08:24.121220 21370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:08:24.126114 21375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:08:24.126206 21370 server_base.cc:1034] running on GCE node
W20260925 22:08:24.126134 21376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:08:24.126138 21378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:08:24.126541 21370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:08:24.126791 21370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:08:24.127990 21370 hybrid_clock.cc:648] HybridClock initialized: now 1790374104127973 us; error 39 us; skew 500 ppm
Sep 25 22:08:24 dist-test-slave-7mqr krb5kdc[20460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374104, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:08:24.130439 21370 init.cc:377] Logged in from keytab as kudu/127.5.0.67@KRBTEST.COM (short username kudu)
I20260925 22:08:24.130903 21370 webserver.cc:460] Webserver started at http://127.5.0.67:42047/ using document root <none> and password file <none>
I20260925 22:08:24.131127 21370 fs_manager.cc:359] Metadata directory not provided
I20260925 22:08:24.131210 21370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:08:24.132627 21370 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260925 22:08:24.133249 21385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:08:24.133438 21370 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:08:24.133544 21370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/wal
uuid: "370bc5fa7b7445ac9d0d063337dc9587"
format_stamp: "Formatted at 2026-09-25 22:08:03 on dist-test-slave-7mqr"
server_key: "c8941ed9c2ddcc439f622304160133a8"
server_key_iv: "4df5d726ec2659b3e105c4747e153c55"
server_key_version: "encryptionkey@0"
I20260925 22:08:24.133862 21370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:08:24.143635 21370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:08:24.145637 21370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:08:24.145797 21370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:08:24.146011 21370 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:08:24.146298 21370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:08:24.146353 21370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:08:24.146391 21370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:08:24.146432 21370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:08:24.152415 21370 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.67:39851
I20260925 22:08:24.152475 21498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.67:39851 every 8 connection(s)
I20260925 22:08:24.152777 21370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
Sep 25 22:08:24 dist-test-slave-7mqr krb5kdc[20460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374104, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:08:24.160189 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 21370
I20260925 22:08:24.162382 21232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:24.153413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:46275 (local address 127.5.0.126:42695)
0925 22:08:24.153490 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0925 22:08:24.153492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:08:24.153733 (+ 241us) server_negotiation.cc:407] Connection header received
0925 22:08:24.154238 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:08:24.154240 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:08:24.154253 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:08:24.154316 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:08:24.155245 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:24.155752 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:24.156792 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:24.156997 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:24.157085 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:24.159248 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:08:24.159250 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:08:24.159251 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:08:24.159258 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:08:24.160911 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:08:24.161413 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:08:24.161414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:08:24.161414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:08:24.161480 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:08:24.161733 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:08:24.161734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:08:24.161734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:08:24.161852 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:08:24.161930 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 22:08:24.162203 (+ 273us) server_negotiation.cc:299] Negotiation successful
0925 22:08:24.162291 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 22:08:24.162731 21501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:24.153453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.67:46275)
0925 22:08:24.153613 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:24.153647 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 22:08:24.154133 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:08:24.154389 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:08:24.154393 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:08:24.154949 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:08:24.155142 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:24.155146 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:24.155851 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:24.155852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:24.156684 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:24.156686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:24.157093 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:24.157094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:24.157217 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:24.157681 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:08:24.157689 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:08:24.159146 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:08:24.161021 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:08:24.161028 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:08:24.161031 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:08:24.161328 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:08:24.161549 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:08:24.161550 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:08:24.161551 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:08:24.161662 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:08:24.161920 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:08:24.161926 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:08:24.162072 (+ 146us) client_negotiation.cc:769] Sending connection context
0925 22:08:24.162181 (+ 109us) client_negotiation.cc:240] Negotiation successful
0925 22:08:24.162333 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":83,"threads_started":1}
I20260925 22:08:24.163225 21499 heartbeater.cc:347] Connected to a master server at 127.5.0.126:42695
I20260925 22:08:24.163305 21499 heartbeater.cc:464] Registering TS with master...
I20260925 22:08:24.163498 21499 heartbeater.cc:511] Master 127.5.0.126:42695 requested a full tablet report, sending...
I20260925 22:08:24.163942 21041 ts_manager.cc:194] Registered new tserver with Master: 370bc5fa7b7445ac9d0d063337dc9587 (127.5.0.67:39851)
I20260925 22:08:24.164297 21041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.67@KRBTEST.COM'} at 127.5.0.67:46275
I20260925 22:08:24.168234 21503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:24.160711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59270 (local address 127.5.0.126:42695)
0925 22:08:24.160844 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0925 22:08:24.160847 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:08:24.160961 (+ 114us) server_negotiation.cc:407] Connection header received
0925 22:08:24.161053 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:08:24.161054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:08:24.161067 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:08:24.161158 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:08:24.161834 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:24.162537 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:24.163845 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:24.164196 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:24.164291 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:24.165728 (+ 1437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:08:24.165729 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:08:24.165730 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:08:24.165737 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:08:24.166954 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:08:24.167337 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:08:24.167338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:08:24.167338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:08:24.167410 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:08:24.167652 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:08:24.167653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:08:24.167653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:08:24.167772 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:08:24.167995 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0925 22:08:24.168036 (+ 41us) server_negotiation.cc:299] Negotiation successful
0925 22:08:24.168122 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":72,"threads_started":1}
I20260925 22:08:24.169137 5121 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:08:24.171733 21503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:24.169505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59282 (local address 127.5.0.126:42695)
0925 22:08:24.169583 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0925 22:08:24.169589 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 22:08:24.169622 (+ 33us) server_negotiation.cc:407] Connection header received
0925 22:08:24.169692 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:08:24.169693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:08:24.169718 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:08:24.169772 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:08:24.169866 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:24.170324 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:24.171127 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:24.171315 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:24.171371 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:24.171536 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:08:24.171574 (+ 38us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0925 22:08:24.171693 (+ 119us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59282: token expired
Metrics: {"server-negotiator.queue_time_us":54}
W20260925 22:08:24.171799 21503 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59282: token expired
W20260925 22:08:24.171984 20969 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.5.0.126:42695)
I20260925 22:08:24.177352 21503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:24.172231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59298 (local address 127.5.0.126:42695)
0925 22:08:24.172278 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 22:08:24.172280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:08:24.172287 (+ 7us) server_negotiation.cc:407] Connection header received
0925 22:08:24.172332 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:08:24.172333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:08:24.172340 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:08:24.172389 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:08:24.172803 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:24.173222 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:24.173869 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:24.174043 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:24.174105 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:24.175229 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:08:24.175230 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:08:24.175231 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:08:24.175236 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:08:24.176499 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:08:24.176784 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:08:24.176785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:08:24.176785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:08:24.176835 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:08:24.176996 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:08:24.176996 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:08:24.176997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:08:24.177078 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:08:24.177142 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 22:08:24.177224 (+ 82us) server_negotiation.cc:299] Negotiation successful
0925 22:08:24.177276 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:08:24.178371 21041 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59298:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:08:24.179947 21041 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:08:24.182232 21163 tablet_service.cc:1467] Processing CreateTablet for tablet 08f32d3441834064bc997dc036caf356 (DEFAULT_TABLE table=test-table [id=408b16861a02452d9bfe868795e12a25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:08:24.182458 21163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08f32d3441834064bc997dc036caf356. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:08:24.183835 21121 tablet_bootstrap.cc:492] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22: Bootstrap starting.
I20260925 22:08:24.184204 21121 tablet_bootstrap.cc:654] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22: Neither blocks nor log segments found. Creating new log.
I20260925 22:08:24.184825 21121 tablet_bootstrap.cc:492] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22: No bootstrap required, opened a new log
I20260925 22:08:24.184870 21121 ts_tablet_manager.cc:1397] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 22:08:24.185196 21121 raft_consensus.cc:348] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e6a6a4c260409d945be3ac9784d5e8" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 41847 } } peers { permanent_uuid: "370bc5fa7b7445ac9d0d063337dc9587" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 39851 } } peers { permanent_uuid: "38b225e2ae0f4f8e971038cc071def22" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37951 } }
I20260925 22:08:24.185263 21121 raft_consensus.cc:374] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:08:24.185312 21121 raft_consensus.cc:729] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b225e2ae0f4f8e971038cc071def22, State: Initialized, Role: FOLLOWER
I20260925 22:08:24.185372 21121 consensus_queue.cc:260] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e6a6a4c260409d945be3ac9784d5e8" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 41847 } } peers { permanent_uuid: "370bc5fa7b7445ac9d0d063337dc9587" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 39851 } } peers { permanent_uuid: "38b225e2ae0f4f8e971038cc071def22" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37951 } }
I20260925 22:08:24.185475 21121 ts_tablet_manager.cc:1428] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260925 22:08:24.186110 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:24.181877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:39851 (local address 127.0.0.1:32998)
0925 22:08:24.182113 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:24.182124 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:08:24.182174 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:08:24.182582 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:08:24.182583 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:08:24.182585 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:08:24.182633 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:24.182634 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:24.183999 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:24.184000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:24.185037 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:24.185040 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:24.185855 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:24.185856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:24.185913 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:24.185934 (+ 21us) client_negotiation.cc:769] Sending connection context
0925 22:08:24.185995 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 22:08:24.186033 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":45,"thread_start_us":85,"threads_started":1}
I20260925 22:08:24.186230 21507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:24.182156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32998 (local address 127.5.0.67:39851)
0925 22:08:24.182300 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0925 22:08:24.182306 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 22:08:24.182318 (+ 12us) server_negotiation.cc:407] Connection header received
0925 22:08:24.182365 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:08:24.182367 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:08:24.182383 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:08:24.182472 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:08:24.182723 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:24.183850 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:24.185126 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:24.185777 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:24.185854 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:24.185917 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 22:08:24.186087 (+ 170us) server_negotiation.cc:299] Negotiation successful
0925 22:08:24.186129 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":83,"threads_started":1}
I20260925 22:08:24.186546 21236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:24.181877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:41847 (local address 127.0.0.1:45602)
0925 22:08:24.182011 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:24.182020 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:08:24.182057 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:08:24.182531 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:08:24.182532 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:08:24.182534 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:08:24.182581 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:24.182582 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:24.184074 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:24.184074 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:24.185245 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:24.185248 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:24.186288 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:24.186289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:24.186369 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:24.186373 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:08:24.186437 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 22:08:24.186478 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119}
I20260925 22:08:24.186787 21506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:24.182014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45602 (local address 127.5.0.66:41847)
0925 22:08:24.182348 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0925 22:08:24.182352 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:08:24.182365 (+ 13us) server_negotiation.cc:407] Connection header received
0925 22:08:24.182405 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:08:24.182408 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:08:24.182423 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:08:24.182650 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:08:24.182755 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:24.183981 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:24.185350 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:24.186127 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:24.186195 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:24.186256 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 22:08:24.186650 (+ 394us) server_negotiation.cc:299] Negotiation successful
0925 22:08:24.186694 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":122,"threads_started":1}
I20260925 22:08:24.187151 21433 tablet_service.cc:1467] Processing CreateTablet for tablet 08f32d3441834064bc997dc036caf356 (DEFAULT_TABLE table=test-table [id=408b16861a02452d9bfe868795e12a25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:08:24.187460 21433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08f32d3441834064bc997dc036caf356. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:08:24.188474 21301 tablet_service.cc:1467] Processing CreateTablet for tablet 08f32d3441834064bc997dc036caf356 (DEFAULT_TABLE table=test-table [id=408b16861a02452d9bfe868795e12a25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:08:24.188747 21301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08f32d3441834064bc997dc036caf356. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:08:24.190214 21509 tablet_bootstrap.cc:492] T 08f32d3441834064bc997dc036caf356 P 370bc5fa7b7445ac9d0d063337dc9587: Bootstrap starting.
I20260925 22:08:24.190976 21509 tablet_bootstrap.cc:654] T 08f32d3441834064bc997dc036caf356 P 370bc5fa7b7445ac9d0d063337dc9587: Neither blocks nor log segments found. Creating new log.
I20260925 22:08:24.191393 21510 tablet_bootstrap.cc:492] T 08f32d3441834064bc997dc036caf356 P 69e6a6a4c260409d945be3ac9784d5e8: Bootstrap starting.
I20260925 22:08:24.191572 21509 log.cc:824] T 08f32d3441834064bc997dc036caf356 P 370bc5fa7b7445ac9d0d063337dc9587: Log is configured to *not* fsync() on all Append() calls
I20260925 22:08:24.192107 21510 tablet_bootstrap.cc:654] T 08f32d3441834064bc997dc036caf356 P 69e6a6a4c260409d945be3ac9784d5e8: Neither blocks nor log segments found. Creating new log.
I20260925 22:08:24.192414 21509 tablet_bootstrap.cc:492] T 08f32d3441834064bc997dc036caf356 P 370bc5fa7b7445ac9d0d063337dc9587: No bootstrap required, opened a new log
I20260925 22:08:24.192500 21509 ts_tablet_manager.cc:1397] T 08f32d3441834064bc997dc036caf356 P 370bc5fa7b7445ac9d0d063337dc9587: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:08:24.192678 21510 log.cc:824] T 08f32d3441834064bc997dc036caf356 P 69e6a6a4c260409d945be3ac9784d5e8: Log is configured to *not* fsync() on all Append() calls
I20260925 22:08:24.193356 21510 tablet_bootstrap.cc:492] T 08f32d3441834064bc997dc036caf356 P 69e6a6a4c260409d945be3ac9784d5e8: No bootstrap required, opened a new log
I20260925 22:08:24.193424 21510 ts_tablet_manager.cc:1397] T 08f32d3441834064bc997dc036caf356 P 69e6a6a4c260409d945be3ac9784d5e8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:08:24.194199 21509 raft_consensus.cc:348] T 08f32d3441834064bc997dc036caf356 P 370bc5fa7b7445ac9d0d063337dc9587 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e6a6a4c260409d945be3ac9784d5e8" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 41847 } } peers { permanent_uuid: "370bc5fa7b7445ac9d0d063337dc9587" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 39851 } } peers { permanent_uuid: "38b225e2ae0f4f8e971038cc071def22" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37951 } }
I20260925 22:08:24.194329 21509 raft_consensus.cc:374] T 08f32d3441834064bc997dc036caf356 P 370bc5fa7b7445ac9d0d063337dc9587 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:08:24.194350 21509 raft_consensus.cc:729] T 08f32d3441834064bc997dc036caf356 P 370bc5fa7b7445ac9d0d063337dc9587 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 370bc5fa7b7445ac9d0d063337dc9587, State: Initialized, Role: FOLLOWER
I20260925 22:08:24.194468 21509 consensus_queue.cc:260] T 08f32d3441834064bc997dc036caf356 P 370bc5fa7b7445ac9d0d063337dc9587 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e6a6a4c260409d945be3ac9784d5e8" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 41847 } } peers { permanent_uuid: "370bc5fa7b7445ac9d0d063337dc9587" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 39851 } } peers { permanent_uuid: "38b225e2ae0f4f8e971038cc071def22" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37951 } }
I20260925 22:08:24.194744 21509 ts_tablet_manager.cc:1428] T 08f32d3441834064bc997dc036caf356 P 370bc5fa7b7445ac9d0d063337dc9587: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 22:08:24.194777 21499 heartbeater.cc:503] Master 127.5.0.126:42695 was elected leader, sending a full tablet report...
I20260925 22:08:24.194964 21510 raft_consensus.cc:348] T 08f32d3441834064bc997dc036caf356 P 69e6a6a4c260409d945be3ac9784d5e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e6a6a4c260409d945be3ac9784d5e8" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 41847 } } peers { permanent_uuid: "370bc5fa7b7445ac9d0d063337dc9587" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 39851 } } peers { permanent_uuid: "38b225e2ae0f4f8e971038cc071def22" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37951 } }
I20260925 22:08:24.195113 21510 raft_consensus.cc:374] T 08f32d3441834064bc997dc036caf356 P 69e6a6a4c260409d945be3ac9784d5e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:08:24.195158 21510 raft_consensus.cc:729] T 08f32d3441834064bc997dc036caf356 P 69e6a6a4c260409d945be3ac9784d5e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69e6a6a4c260409d945be3ac9784d5e8, State: Initialized, Role: FOLLOWER
I20260925 22:08:24.195272 21510 consensus_queue.cc:260] T 08f32d3441834064bc997dc036caf356 P 69e6a6a4c260409d945be3ac9784d5e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e6a6a4c260409d945be3ac9784d5e8" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 41847 } } peers { permanent_uuid: "370bc5fa7b7445ac9d0d063337dc9587" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 39851 } } peers { permanent_uuid: "38b225e2ae0f4f8e971038cc071def22" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37951 } }
I20260925 22:08:24.195571 21367 heartbeater.cc:503] Master 127.5.0.126:42695 was elected leader, sending a full tablet report...
I20260925 22:08:24.195554 21510 ts_tablet_manager.cc:1428] T 08f32d3441834064bc997dc036caf356 P 69e6a6a4c260409d945be3ac9784d5e8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260925 22:08:24.277211 21368 tablet.cc:2369] T 08f32d3441834064bc997dc036caf356 P 69e6a6a4c260409d945be3ac9784d5e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:08:24.309031 21513 raft_consensus.cc:482] T 08f32d3441834064bc997dc036caf356 P 370bc5fa7b7445ac9d0d063337dc9587 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:08:24.309185 21513 raft_consensus.cc:504] T 08f32d3441834064bc997dc036caf356 P 370bc5fa7b7445ac9d0d063337dc9587 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e6a6a4c260409d945be3ac9784d5e8" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 41847 } } peers { permanent_uuid: "370bc5fa7b7445ac9d0d063337dc9587" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 39851 } } peers { permanent_uuid: "38b225e2ae0f4f8e971038cc071def22" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37951 } }
I20260925 22:08:24.309561 21513 leader_election.cc:288] T 08f32d3441834064bc997dc036caf356 P 370bc5fa7b7445ac9d0d063337dc9587 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 69e6a6a4c260409d945be3ac9784d5e8 (127.5.0.66:41847), 38b225e2ae0f4f8e971038cc071def22 (127.5.0.65:37951)
I20260925 22:08:24.312798 21237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:24.310071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:36849 (local address 127.5.0.65:37951)
0925 22:08:24.310119 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 22:08:24.310121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:08:24.310260 (+ 139us) server_negotiation.cc:407] Connection header received
0925 22:08:24.310336 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:08:24.310337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:08:24.310368 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:08:24.310476 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:08:24.310612 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:24.311100 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:24.312071 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:24.312475 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:24.312562 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:24.312666 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 22:08:24.312686 (+ 20us) server_negotiation.cc:299] Negotiation successful
0925 22:08:24.312719 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 22:08:24.312904 21501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:24.309845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:41847 (local address 127.5.0.67:54299)
0925 22:08:24.309960 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:24.309975 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 22:08:24.310045 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:08:24.310239 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:08:24.310240 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:08:24.310242 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:08:24.310326 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:24.310328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:24.311054 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:24.311055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:24.311929 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:24.311931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:24.312541 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:24.312542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:24.312655 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:24.312659 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:08:24.312713 (+ 54us) client_negotiation.cc:240] Negotiation successful
0925 22:08:24.312752 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":32}
I20260925 22:08:24.312964 21506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:24.309982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:54299 (local address 127.5.0.66:41847)
0925 22:08:24.310036 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 22:08:24.310038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:08:24.310050 (+ 12us) server_negotiation.cc:407] Connection header received
0925 22:08:24.310142 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:08:24.310143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:08:24.310155 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:08:24.310235 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:08:24.310409 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:24.310958 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:24.312032 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:24.312443 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:24.312531 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:24.312542 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0925 22:08:24.312812 (+ 270us) server_negotiation.cc:299] Negotiation successful
0925 22:08:24.312876 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 22:08:24.313043 21515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:24.310001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:37951 (local address 127.5.0.67:36849)
0925 22:08:24.310182 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:24.310190 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 22:08:24.310268 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:08:24.310444 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:08:24.310445 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:08:24.310446 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:08:24.310525 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:24.310526 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:24.311183 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:24.311184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:24.311992 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:24.311994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:24.312562 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:24.312563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:24.312629 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:24.312633 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:08:24.312677 (+ 44us) client_negotiation.cc:240] Negotiation successful
0925 22:08:24.312715 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":2,"thread_start_us":114,"threads_started":1}
I20260925 22:08:24.313364 21321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f32d3441834064bc997dc036caf356" candidate_uuid: "370bc5fa7b7445ac9d0d063337dc9587" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69e6a6a4c260409d945be3ac9784d5e8" is_pre_election: true
I20260925 22:08:24.313551 21321 raft_consensus.cc:2454] T 08f32d3441834064bc997dc036caf356 P 69e6a6a4c260409d945be3ac9784d5e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 370bc5fa7b7445ac9d0d063337dc9587 in term 0.
I20260925 22:08:24.313791 21389 leader_election.cc:302] T 08f32d3441834064bc997dc036caf356 P 370bc5fa7b7445ac9d0d063337dc9587 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 370bc5fa7b7445ac9d0d063337dc9587, 69e6a6a4c260409d945be3ac9784d5e8; no voters:
I20260925 22:08:24.313768 21183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f32d3441834064bc997dc036caf356" candidate_uuid: "370bc5fa7b7445ac9d0d063337dc9587" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b225e2ae0f4f8e971038cc071def22" is_pre_election: true
I20260925 22:08:24.313967 21183 raft_consensus.cc:2454] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 370bc5fa7b7445ac9d0d063337dc9587 in term 0.
I20260925 22:08:24.313959 21513 raft_consensus.cc:2781] T 08f32d3441834064bc997dc036caf356 P 370bc5fa7b7445ac9d0d063337dc9587 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:08:24.314055 21513 raft_consensus.cc:482] T 08f32d3441834064bc997dc036caf356 P 370bc5fa7b7445ac9d0d063337dc9587 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:08:24.314113 21513 raft_consensus.cc:3037] T 08f32d3441834064bc997dc036caf356 P 370bc5fa7b7445ac9d0d063337dc9587 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:08:24.314926 21513 raft_consensus.cc:504] T 08f32d3441834064bc997dc036caf356 P 370bc5fa7b7445ac9d0d063337dc9587 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e6a6a4c260409d945be3ac9784d5e8" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 41847 } } peers { permanent_uuid: "370bc5fa7b7445ac9d0d063337dc9587" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 39851 } } peers { permanent_uuid: "38b225e2ae0f4f8e971038cc071def22" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37951 } }
I20260925 22:08:24.315068 21513 leader_election.cc:288] T 08f32d3441834064bc997dc036caf356 P 370bc5fa7b7445ac9d0d063337dc9587 [CANDIDATE]: Term 1 election: Requested vote from peers 69e6a6a4c260409d945be3ac9784d5e8 (127.5.0.66:41847), 38b225e2ae0f4f8e971038cc071def22 (127.5.0.65:37951)
I20260925 22:08:24.315238 21321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f32d3441834064bc997dc036caf356" candidate_uuid: "370bc5fa7b7445ac9d0d063337dc9587" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69e6a6a4c260409d945be3ac9784d5e8"
I20260925 22:08:24.315336 21321 raft_consensus.cc:3037] T 08f32d3441834064bc997dc036caf356 P 69e6a6a4c260409d945be3ac9784d5e8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:08:24.315408 21183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f32d3441834064bc997dc036caf356" candidate_uuid: "370bc5fa7b7445ac9d0d063337dc9587" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b225e2ae0f4f8e971038cc071def22"
I20260925 22:08:24.315490 21183 raft_consensus.cc:3037] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:08:24.316462 21183 raft_consensus.cc:2454] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 370bc5fa7b7445ac9d0d063337dc9587 in term 1.
I20260925 22:08:24.316462 21321 raft_consensus.cc:2454] T 08f32d3441834064bc997dc036caf356 P 69e6a6a4c260409d945be3ac9784d5e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 370bc5fa7b7445ac9d0d063337dc9587 in term 1.
I20260925 22:08:24.316637 21389 leader_election.cc:302] T 08f32d3441834064bc997dc036caf356 P 370bc5fa7b7445ac9d0d063337dc9587 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 370bc5fa7b7445ac9d0d063337dc9587, 38b225e2ae0f4f8e971038cc071def22; no voters:
I20260925 22:08:24.316738 21513 raft_consensus.cc:2781] T 08f32d3441834064bc997dc036caf356 P 370bc5fa7b7445ac9d0d063337dc9587 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:08:24.316887 21513 raft_consensus.cc:686] T 08f32d3441834064bc997dc036caf356 P 370bc5fa7b7445ac9d0d063337dc9587 [term 1 LEADER]: Becoming Leader. State: Replica: 370bc5fa7b7445ac9d0d063337dc9587, State: Running, Role: LEADER
I20260925 22:08:24.316985 21513 consensus_queue.cc:237] T 08f32d3441834064bc997dc036caf356 P 370bc5fa7b7445ac9d0d063337dc9587 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e6a6a4c260409d945be3ac9784d5e8" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 41847 } } peers { permanent_uuid: "370bc5fa7b7445ac9d0d063337dc9587" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 39851 } } peers { permanent_uuid: "38b225e2ae0f4f8e971038cc071def22" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37951 } }
I20260925 22:08:24.317816 21040 catalog_manager.cc:5495] T 08f32d3441834064bc997dc036caf356 P 370bc5fa7b7445ac9d0d063337dc9587 reported cstate change: term changed from 0 to 1, leader changed from <none> to 370bc5fa7b7445ac9d0d063337dc9587 (127.5.0.67). New cstate: current_term: 1 leader_uuid: "370bc5fa7b7445ac9d0d063337dc9587" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e6a6a4c260409d945be3ac9784d5e8" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 41847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "370bc5fa7b7445ac9d0d063337dc9587" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 39851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38b225e2ae0f4f8e971038cc071def22" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37951 } health_report { overall_health: UNKNOWN } } }
I20260925 22:08:24.333196 21507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:24.330639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33000 (local address 127.5.0.67:39851)
0925 22:08:24.330698 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 22:08:24.330700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:08:24.330729 (+ 29us) server_negotiation.cc:407] Connection header received
0925 22:08:24.330806 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:08:24.330807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:08:24.330826 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:08:24.330910 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:08:24.331053 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:24.331542 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:24.332400 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:24.332602 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:24.332684 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:24.332836 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:08:24.332936 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:08:24.333024 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 22:08:24.333065 (+ 41us) server_negotiation.cc:299] Negotiation successful
0925 22:08:24.333086 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 22:08:24.335047 21183 raft_consensus.cc:1261] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [term 1 FOLLOWER]: Refusing update from remote peer 370bc5fa7b7445ac9d0d063337dc9587: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:08:24.335062 21321 raft_consensus.cc:1261] T 08f32d3441834064bc997dc036caf356 P 69e6a6a4c260409d945be3ac9784d5e8 [term 1 FOLLOWER]: Refusing update from remote peer 370bc5fa7b7445ac9d0d063337dc9587: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:08:24.335421 21513 consensus_queue.cc:1035] T 08f32d3441834064bc997dc036caf356 P 370bc5fa7b7445ac9d0d063337dc9587 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b225e2ae0f4f8e971038cc071def22" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:08:24.335548 21516 consensus_queue.cc:1035] T 08f32d3441834064bc997dc036caf356 P 370bc5fa7b7445ac9d0d063337dc9587 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69e6a6a4c260409d945be3ac9784d5e8" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 41847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:08:24.337970 21520 mvcc.cc:205] Tried to move back new op lower bound from 7333372331353530368 to 7333372331283005440. Current Snapshot: MvccSnapshot[applied={T|T < 7333372331353530368}]
I20260925 22:08:24.337985 21519 mvcc.cc:205] Tried to move back new op lower bound from 7333372331353530368 to 7333372331283005440. Current Snapshot: MvccSnapshot[applied={T|T < 7333372331353530368}]
I20260925 22:08:24.338266 21518 mvcc.cc:205] Tried to move back new op lower bound from 7333372331353530368 to 7333372331283005440. Current Snapshot: MvccSnapshot[applied={T|T < 7333372331353530368 or (T in {7333372331353530368})}]
I20260925 22:08:24.339954 21041 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59298:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:08:24.340050 21041 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59298:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:08:24.340830 21041 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d: Sending DeleteTablet for 3 replicas of tablet 08f32d3441834064bc997dc036caf356
I20260925 22:08:24.341143 21433 tablet_service.cc:1514] Processing DeleteTablet for tablet 08f32d3441834064bc997dc036caf356 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:08:24 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:32998
I20260925 22:08:24.341161 21301 tablet_service.cc:1514] Processing DeleteTablet for tablet 08f32d3441834064bc997dc036caf356 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:08:24 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:45602
I20260925 22:08:24.341209 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 21079
I20260925 22:08:24.341322 21163 tablet_service.cc:1514] Processing DeleteTablet for tablet 08f32d3441834064bc997dc036caf356 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:08:24 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:41558
I20260925 22:08:24.341862 21528 tablet_replica.cc:330] T 08f32d3441834064bc997dc036caf356 P 69e6a6a4c260409d945be3ac9784d5e8: stopping tablet replica
I20260925 22:08:24.341979 21528 raft_consensus.cc:2229] T 08f32d3441834064bc997dc036caf356 P 69e6a6a4c260409d945be3ac9784d5e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:08:24.342074 21528 raft_consensus.cc:2258] T 08f32d3441834064bc997dc036caf356 P 69e6a6a4c260409d945be3ac9784d5e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:08:24.342221 21529 tablet_replica.cc:330] T 08f32d3441834064bc997dc036caf356 P 370bc5fa7b7445ac9d0d063337dc9587: stopping tablet replica
I20260925 22:08:24.342308 21529 raft_consensus.cc:2229] T 08f32d3441834064bc997dc036caf356 P 370bc5fa7b7445ac9d0d063337dc9587 [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:08:24.342393 21529 raft_consensus.cc:2258] T 08f32d3441834064bc997dc036caf356 P 370bc5fa7b7445ac9d0d063337dc9587 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:08:24.342785 21529 ts_tablet_manager.cc:1905] T 08f32d3441834064bc997dc036caf356 P 370bc5fa7b7445ac9d0d063337dc9587: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:08:24.342785 21528 ts_tablet_manager.cc:1905] T 08f32d3441834064bc997dc036caf356 P 69e6a6a4c260409d945be3ac9784d5e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:08:24.344530 21528 ts_tablet_manager.cc:1918] T 08f32d3441834064bc997dc036caf356 P 69e6a6a4c260409d945be3ac9784d5e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:08:24.344628 21528 log.cc:1196] T 08f32d3441834064bc997dc036caf356 P 69e6a6a4c260409d945be3ac9784d5e8: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/wal/wals/08f32d3441834064bc997dc036caf356
I20260925 22:08:24.344767 21529 ts_tablet_manager.cc:1918] T 08f32d3441834064bc997dc036caf356 P 370bc5fa7b7445ac9d0d063337dc9587: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:08:24.344887 21529 log.cc:1196] T 08f32d3441834064bc997dc036caf356 P 370bc5fa7b7445ac9d0d063337dc9587: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/wal/wals/08f32d3441834064bc997dc036caf356
I20260925 22:08:24.344931 21528 ts_tablet_manager.cc:1939] T 08f32d3441834064bc997dc036caf356 P 69e6a6a4c260409d945be3ac9784d5e8: Deleting consensus metadata
I20260925 22:08:24.345239 21529 ts_tablet_manager.cc:1939] T 08f32d3441834064bc997dc036caf356 P 370bc5fa7b7445ac9d0d063337dc9587: Deleting consensus metadata
I20260925 22:08:24.345352 21028 catalog_manager.cc:4841] TS 69e6a6a4c260409d945be3ac9784d5e8 (127.5.0.66:41847): tablet 08f32d3441834064bc997dc036caf356 (table test-table [id=408b16861a02452d9bfe868795e12a25]) successfully deleted
I20260925 22:08:24.348026 21025 catalog_manager.cc:4841] TS 370bc5fa7b7445ac9d0d063337dc9587 (127.5.0.67:39851): tablet 08f32d3441834064bc997dc036caf356 (table test-table [id=408b16861a02452d9bfe868795e12a25]) successfully deleted
W20260925 22:08:24.348675 21028 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.0.65:37951 (error 108)
I20260925 22:08:24.349120 21236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:24.348981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:37951 (local address 127.0.0.1:41568)
0925 22:08:24.349043 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:24.349084 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.65:37951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260925 22:08:24.349130 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 21234
W20260925 22:08:24.352475 21028 catalog_manager.cc:4584] TS 38b225e2ae0f4f8e971038cc071def22 (127.5.0.65:37951): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 08f32d3441834064bc997dc036caf356: Network error: Client connection negotiation failed: client connection to 127.5.0.65:37951: connect: Connection refused (error 111)
I20260925 22:08:24.354657 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 21370
I20260925 22:08:24.359100 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 21008
I20260925 22:08:44.364367 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:42695
--webserver_interface=127.5.0.126
--webserver_port=37595
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:37557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:42695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:08:44.444550 21531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:08:44.444756 21531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:08:44.444777 21531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:08:44.445860 21531 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 22:08:44.445894 21531 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 22:08:44.445906 21531 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 22:08:44.445917 21531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:08:44.445930 21531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:08:44.445943 21531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:08:44.445955 21531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:08:44.445966 21531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:08:44.447343 21531 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:37557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:42695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:42695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=37595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:08:44.447543 21531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:08:44.447705 21531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:08:44.452826 21536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:08:44.452886 21531 server_base.cc:1034] running on GCE node
W20260925 22:08:44.452875 21539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:08:44.452836 21537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:08:44.453320 21531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:08:44.453639 21531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:08:44.454816 21531 hybrid_clock.cc:648] HybridClock initialized: now 1790374124454793 us; error 47 us; skew 500 ppm
Sep 25 22:08:44 dist-test-slave-7mqr krb5kdc[20460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374124, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:08:44.457367 21531 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:08:44.457823 21531 webserver.cc:460] Webserver started at http://127.5.0.126:37595/ using document root <none> and password file <none>
I20260925 22:08:44.458057 21531 fs_manager.cc:359] Metadata directory not provided
I20260925 22:08:44.458127 21531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:08:44.459554 21531 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:08:44.460309 21546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:08:44.460484 21531 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 22:08:44.460587 21531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "521bc070528545ebb74dce96dfc41a2d"
format_stamp: "Formatted at 2026-09-25 22:08:02 on dist-test-slave-7mqr"
server_key: "0ad726727fd5c79765dbef07dc511f23"
server_key_iv: "166fd940d02c1fc69bd05205333f223b"
server_key_version: "encryptionkey@0"
I20260925 22:08:44.460958 21531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:08:44.473711 21531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:08:44.476060 21531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:08:44.476217 21531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:08:44.480022 21531 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:42695
I20260925 22:08:44.480075 21598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:42695 every 8 connection(s)
I20260925 22:08:44.480454 21531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:08:44.481904 21599 sys_catalog.cc:263] Verifying existing consensus state
I20260925 22:08:44.482569 21599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d: Bootstrap starting.
I20260925 22:08:44.489513 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 21531
I20260925 22:08:44.489832 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.65:37951
--local_ip_for_outbound_sockets=127.5.0.65
--tserver_master_addrs=127.5.0.126:42695
--webserver_port=42041
--webserver_interface=127.5.0.65
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:37557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:08:44.490921 21599 log.cc:824] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d: Log is configured to *not* fsync() on all Append() calls
I20260925 22:08:44.493275 21599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 22:08:44.493575 21599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d: Bootstrap complete.
I20260925 22:08:44.495746 21599 raft_consensus.cc:348] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521bc070528545ebb74dce96dfc41a2d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42695 } }
I20260925 22:08:44.496059 21599 raft_consensus.cc:729] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 521bc070528545ebb74dce96dfc41a2d, State: Initialized, Role: FOLLOWER
I20260925 22:08:44.496236 21599 consensus_queue.cc:260] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521bc070528545ebb74dce96dfc41a2d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42695 } }
I20260925 22:08:44.496297 21599 raft_consensus.cc:388] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:08:44.496357 21599 raft_consensus.cc:482] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:08:44.496407 21599 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [term 2 FOLLOWER]: Advancing to term 3
I20260925 22:08:44.498217 21599 raft_consensus.cc:504] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521bc070528545ebb74dce96dfc41a2d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42695 } }
I20260925 22:08:44.498304 21599 leader_election.cc:302] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 521bc070528545ebb74dce96dfc41a2d; no voters:
I20260925 22:08:44.498482 21599 leader_election.cc:288] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [CANDIDATE]: Term 3 election: Requested vote from peers
I20260925 22:08:44.498546 21603 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [term 3 FOLLOWER]: Leader election won for term 3
I20260925 22:08:44.498673 21603 raft_consensus.cc:686] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [term 3 LEADER]: Becoming Leader. State: Replica: 521bc070528545ebb74dce96dfc41a2d, State: Running, Role: LEADER
I20260925 22:08:44.498751 21599 sys_catalog.cc:564] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:08:44.498760 21603 consensus_queue.cc:237] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521bc070528545ebb74dce96dfc41a2d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42695 } }
I20260925 22:08:44.499187 21605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 521bc070528545ebb74dce96dfc41a2d. Latest consensus state: current_term: 3 leader_uuid: "521bc070528545ebb74dce96dfc41a2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521bc070528545ebb74dce96dfc41a2d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42695 } } }
I20260925 22:08:44.499294 21605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [sys.catalog]: This master's current role is: LEADER
I20260925 22:08:44.499204 21604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "521bc070528545ebb74dce96dfc41a2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521bc070528545ebb74dce96dfc41a2d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42695 } } }
I20260925 22:08:44.499444 21604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d [sys.catalog]: This master's current role is: LEADER
I20260925 22:08:44.499583 21612 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:08:44.500469 21612 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:08:44.500597 21612 catalog_manager.cc:1259] Loaded cluster ID: 92e572e87e274bee8b613af0ca1dfb9e
I20260925 22:08:44.500804 21612 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:08:44.501592 21612 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:08:44.501771 21612 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d: Loaded TSK: 0
I20260925 22:08:44.501809 21612 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 521bc070528545ebb74dce96dfc41a2d: Loaded TSK: 1
F20260925 22:08:44.502038 21612 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f870d1979ad google::LogMessage::Fail() at ??:0
@ 0x7f870d19a88a google::LogMessage::SendToLog() at ??:0
@ 0x7f870d1974de google::LogMessage::Flush() at ??:0
@ 0x7f870d198a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f870f904104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f870f901d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f87150f51cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f87150f548e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f87150f5594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f87150bbc48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f87150c5ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f870f0650d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f870f05a954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f870febe6db start_thread at ??:0
@ 0x7f870bc9561f clone at ??:0
W20260925 22:08:44.591384 21601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:08:44.591585 21601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:08:44.591609 21601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:08:44.592798 21601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:08:44.592841 21601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:08:44.592866 21601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.65
I20260925 22:08:44.594287 21601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:37557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.65:37951
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.0.65
--webserver_port=42041
--tserver_master_addrs=127.5.0.126:42695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.65
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:08:44.594540 21601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:08:44.594766 21601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:08:44.599603 21641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:08:44.599733 21640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:08:44.599828 21601 server_base.cc:1034] running on GCE node
W20260925 22:08:44.599592 21643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:08:44.600163 21601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:08:44.600441 21601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:08:44.601603 21601 hybrid_clock.cc:648] HybridClock initialized: now 1790374124601587 us; error 36 us; skew 500 ppm
Sep 25 22:08:44 dist-test-slave-7mqr krb5kdc[20460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374124, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:08:44.604202 21601 init.cc:377] Logged in from keytab as kudu/127.5.0.65@KRBTEST.COM (short username kudu)
I20260925 22:08:44.604606 21601 webserver.cc:460] Webserver started at http://127.5.0.65:42041/ using document root <none> and password file <none>
I20260925 22:08:44.604839 21601 fs_manager.cc:359] Metadata directory not provided
I20260925 22:08:44.604902 21601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:08:44.606249 21601 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:08:44.606980 21650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:08:44.607148 21601 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:08:44.607245 21601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/wal
uuid: "38b225e2ae0f4f8e971038cc071def22"
format_stamp: "Formatted at 2026-09-25 22:08:02 on dist-test-slave-7mqr"
server_key: "938f1b6a9bedd22127b0db309cfa085d"
server_key_iv: "94b871fada6e304e219c5560df27ead0"
server_key_version: "encryptionkey@0"
I20260925 22:08:44.607573 21601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:08:44.620743 21601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:08:44.622934 21601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:08:44.623082 21601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:08:44.623296 21601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:08:44.623709 21657 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 22:08:44.625717 21601 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 22:08:44.625778 21601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260925 22:08:44.625821 21601 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 22:08:44.626462 21601 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 22:08:44.626516 21601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260925 22:08:44.626590 21657 tablet_bootstrap.cc:492] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22: Bootstrap starting.
I20260925 22:08:44.633899 21601 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.65:37951
I20260925 22:08:44.634357 21601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
I20260925 22:08:44.634912 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 21601
I20260925 22:08:44.635258 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.66:41847
--local_ip_for_outbound_sockets=127.5.0.66
--tserver_master_addrs=127.5.0.126:42695
--webserver_port=45095
--webserver_interface=127.5.0.66
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:37557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:08:44.637364 21657 log.cc:824] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22: Log is configured to *not* fsync() on all Append() calls
I20260925 22:08:44.639072 21764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.65:37951 every 8 connection(s)
I20260925 22:08:44.639605 21657 tablet_bootstrap.cc:492] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 22:08:44.639935 21657 tablet_bootstrap.cc:492] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22: Bootstrap complete.
I20260925 22:08:44.641120 21657 ts_tablet_manager.cc:1397] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.010s
I20260925 22:08:44.642882 21657 raft_consensus.cc:348] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e6a6a4c260409d945be3ac9784d5e8" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 41847 } } peers { permanent_uuid: "370bc5fa7b7445ac9d0d063337dc9587" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 39851 } } peers { permanent_uuid: "38b225e2ae0f4f8e971038cc071def22" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37951 } }
I20260925 22:08:44.643283 21657 raft_consensus.cc:729] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b225e2ae0f4f8e971038cc071def22, State: Initialized, Role: FOLLOWER
I20260925 22:08:44.643432 21657 consensus_queue.cc:260] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e6a6a4c260409d945be3ac9784d5e8" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 41847 } } peers { permanent_uuid: "370bc5fa7b7445ac9d0d063337dc9587" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 39851 } } peers { permanent_uuid: "38b225e2ae0f4f8e971038cc071def22" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37951 } }
I20260925 22:08:44.643728 21657 ts_tablet_manager.cc:1428] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22: Time spent starting tablet: real 0.003s user 0.001s sys 0.003s
W20260925 22:08:44.694599 21767 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:08:44.635511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.65:41731)
0925 22:08:44.636015 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:44.636063 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 22:08:44.636813 (+ 750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:08:44.694380 (+ 57567us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":93,"threads_started":1}
W20260925 22:08:44.694911 21765 heartbeater.cc:644] Failed to heartbeat to 127.5.0.126:42695 (0 consecutive failures): Network error: Failed to ping master at 127.5.0.126:42695: Client connection negotiation failed: client connection to 127.5.0.126:42695: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 22:08:44.695353 21767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:44.695199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.65:42047)
0925 22:08:44.695284 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:44.695329 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260925 22:08:44.695744 21767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:44.695642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.65:37273)
0925 22:08:44.695691 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:44.695718 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260925 22:08:44.695852 21765 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 22:08:44.723779 21768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:08:44.724076 21768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:08:44.724116 21768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:08:44.725306 21768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:08:44.725370 21768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:08:44.725415 21768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.66
I20260925 22:08:44.726878 21768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:37557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.66:41847
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.0.66
--webserver_port=45095
--tserver_master_addrs=127.5.0.126:42695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.66
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:08:44.727145 21768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:08:44.727406 21768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 22:08:44.732806 21768 server_base.cc:1034] running on GCE node
W20260925 22:08:44.732790 21778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:08:44.732781 21776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:08:44.732790 21775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:08:44.733214 21768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:08:44.733409 21768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:08:44.734613 21768 hybrid_clock.cc:648] HybridClock initialized: now 1790374124734616 us; error 57 us; skew 500 ppm
Sep 25 22:08:44 dist-test-slave-7mqr krb5kdc[20460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374124, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:08:44.737201 21768 init.cc:377] Logged in from keytab as kudu/127.5.0.66@KRBTEST.COM (short username kudu)
I20260925 22:08:44.737730 21768 webserver.cc:460] Webserver started at http://127.5.0.66:45095/ using document root <none> and password file <none>
I20260925 22:08:44.737962 21768 fs_manager.cc:359] Metadata directory not provided
I20260925 22:08:44.738030 21768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:08:44.739513 21768 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:08:44.740176 21785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:08:44.740350 21768 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:08:44.740460 21768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/wal
uuid: "69e6a6a4c260409d945be3ac9784d5e8"
format_stamp: "Formatted at 2026-09-25 22:08:03 on dist-test-slave-7mqr"
server_key: "a0a60a847f3afade7b4469af792b3ab1"
server_key_iv: "90cf14e5e90db0adf0eba136598352d6"
server_key_version: "encryptionkey@0"
I20260925 22:08:44.740803 21768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:08:44.780393 21768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:08:44.782440 21768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:08:44.782611 21768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:08:44.782871 21768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:08:44.783188 21768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:08:44.783242 21768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:08:44.783280 21768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:08:44.783329 21768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:08:44.789373 21768 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.66:41847
I20260925 22:08:44.789417 21898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.66:41847 every 8 connection(s)
I20260925 22:08:44.789764 21768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
I20260925 22:08:44.790839 21901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:44.790399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.66:51257)
0925 22:08:44.790554 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:44.790647 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":74,"threads_started":1}
W20260925 22:08:44.791447 21899 heartbeater.cc:644] Failed to heartbeat to 127.5.0.126:42695 (0 consecutive failures): Network error: Failed to ping master at 127.5.0.126:42695: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
I20260925 22:08:44.791779 21901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:44.791696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.66:43809)
0925 22:08:44.791735 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:44.791760 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260925 22:08:44.792238 21901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:44.792119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.66:60453)
0925 22:08:44.792185 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:44.792219 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260925 22:08:44.792392 21899 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 22:08:44.793138 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 21768
I20260925 22:08:44.793433 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.67:39851
--local_ip_for_outbound_sockets=127.5.0.67
--tserver_master_addrs=127.5.0.126:42695
--webserver_port=42047
--webserver_interface=127.5.0.67
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:37557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:08:44.874297 21902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:08:44.874500 21902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:08:44.874522 21902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:08:44.875634 21902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:08:44.875675 21902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:08:44.875699 21902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.67
I20260925 22:08:44.877220 21902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:37557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.67:39851
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.0.67
--webserver_port=42047
--tserver_master_addrs=127.5.0.126:42695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.67
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:08:44.877485 21902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:08:44.877712 21902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:08:44.882827 21907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:08:44.882835 21910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:08:44.882835 21908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:08:44.883229 21902 server_base.cc:1034] running on GCE node
I20260925 22:08:44.883453 21902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:08:44.883710 21902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:08:44.884892 21902 hybrid_clock.cc:648] HybridClock initialized: now 1790374124884879 us; error 32 us; skew 500 ppm
Sep 25 22:08:44 dist-test-slave-7mqr krb5kdc[20460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374124, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:08:44.887552 21902 init.cc:377] Logged in from keytab as kudu/127.5.0.67@KRBTEST.COM (short username kudu)
I20260925 22:08:44.888093 21902 webserver.cc:460] Webserver started at http://127.5.0.67:42047/ using document root <none> and password file <none>
I20260925 22:08:44.888286 21902 fs_manager.cc:359] Metadata directory not provided
I20260925 22:08:44.888327 21902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:08:44.889573 21902 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:08:44.890396 21917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:08:44.890612 21902 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:08:44.890708 21902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/wal
uuid: "370bc5fa7b7445ac9d0d063337dc9587"
format_stamp: "Formatted at 2026-09-25 22:08:03 on dist-test-slave-7mqr"
server_key: "c8941ed9c2ddcc439f622304160133a8"
server_key_iv: "4df5d726ec2659b3e105c4747e153c55"
server_key_version: "encryptionkey@0"
I20260925 22:08:44.891057 21902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:08:44.911580 21902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:08:44.913553 21902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:08:44.913681 21902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:08:44.913862 21902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:08:44.914201 21902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:08:44.914242 21902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:08:44.914264 21902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:08:44.914278 21902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:08:44.920017 21902 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.67:39851
I20260925 22:08:44.920095 22030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.67:39851 every 8 connection(s)
I20260925 22:08:44.920449 21902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
I20260925 22:08:44.921530 22033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:44.921168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.67:34347)
0925 22:08:44.921313 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:44.921384 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":68,"threads_started":1}
W20260925 22:08:44.921945 22031 heartbeater.cc:644] Failed to heartbeat to 127.5.0.126:42695 (0 consecutive failures): Network error: Failed to ping master at 127.5.0.126:42695: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
I20260925 22:08:44.922233 22033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:44.922139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.67:59565)
0925 22:08:44.922189 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:44.922215 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260925 22:08:44.922534 22033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:44.922459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.67:58329)
0925 22:08:44.922497 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:44.922519 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260925 22:08:44.922665 22031 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 22:08:44.928707 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 21902
I20260925 22:08:45.696602 22035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:45.696307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.65:38755)
0925 22:08:45.696479 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:45.696532 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":72,"threads_started":1}
I20260925 22:08:45.792956 22036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:45.792731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.66:35223)
0925 22:08:45.792865 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:45.792906 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":57,"threads_started":1}
I20260925 22:08:45.923168 22037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:45.922972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.67:36341)
0925 22:08:45.923086 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:45.923120 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":57,"threads_started":1}
I20260925 22:08:46.048187 22038 raft_consensus.cc:482] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:08:46.048317 22038 raft_consensus.cc:504] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e6a6a4c260409d945be3ac9784d5e8" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 41847 } } peers { permanent_uuid: "370bc5fa7b7445ac9d0d063337dc9587" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 39851 } } peers { permanent_uuid: "38b225e2ae0f4f8e971038cc071def22" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37951 } }
I20260925 22:08:46.048808 22038 leader_election.cc:288] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 69e6a6a4c260409d945be3ac9784d5e8 (127.5.0.66:41847), 370bc5fa7b7445ac9d0d063337dc9587 (127.5.0.67:39851)
Sep 25 22:08:46 dist-test-slave-7mqr krb5kdc[20460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374124, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for kudu/127.5.0.67@KRBTEST.COM
Sep 25 22:08:46 dist-test-slave-7mqr krb5kdc[20460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374124, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for kudu/127.5.0.66@KRBTEST.COM
I20260925 22:08:46.065160 22041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:46.049261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:55895 (local address 127.5.0.67:39851)
0925 22:08:46.049438 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 22:08:46.049447 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 22:08:46.049473 (+ 26us) server_negotiation.cc:407] Connection header received
0925 22:08:46.049951 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:08:46.049954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:08:46.049994 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:08:46.050153 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:08:46.050912 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:46.051783 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:46.052798 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:46.053205 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:46.053331 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:46.061295 (+ 7964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:08:46.061301 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:08:46.061301 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:08:46.061315 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:08:46.063251 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:08:46.063781 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:08:46.063784 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:08:46.063785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:08:46.063924 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:08:46.064222 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:08:46.064223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:08:46.064223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:08:46.064504 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:08:46.064568 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 22:08:46.064885 (+ 317us) server_negotiation.cc:299] Negotiation successful
0925 22:08:46.065036 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":82,"threads_started":1}
I20260925 22:08:46.065147 22035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:46.049022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:41847 (local address 127.5.0.65:47949)
0925 22:08:46.049082 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:46.049095 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:08:46.049152 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:08:46.050190 (+ 1038us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:08:46.050193 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:08:46.050578 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:08:46.050798 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:46.050812 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:46.052051 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:46.052052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:46.052708 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:46.052710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:46.053333 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:46.053334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:46.053411 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:46.059274 (+ 5863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:08:46.059284 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:08:46.061493 (+ 2209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:08:46.063618 (+ 2125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:08:46.063619 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:08:46.063620 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:08:46.063832 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:08:46.064146 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:08:46.064147 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:08:46.064147 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:08:46.064236 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:08:46.064669 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:08:46.064670 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:08:46.064725 (+ 55us) client_negotiation.cc:769] Sending connection context
0925 22:08:46.064835 (+ 110us) client_negotiation.cc:240] Negotiation successful
0925 22:08:46.065050 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":1}
I20260925 22:08:46.065191 22040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:46.049155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:47949 (local address 127.5.0.66:41847)
0925 22:08:46.049283 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0925 22:08:46.049294 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 22:08:46.049323 (+ 29us) server_negotiation.cc:407] Connection header received
0925 22:08:46.049945 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:08:46.049947 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:08:46.049994 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:08:46.050163 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:08:46.050922 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:46.051858 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:46.052812 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:08:46.053238 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:08:46.053355 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:46.061625 (+ 8270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:08:46.061629 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:08:46.061629 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:08:46.061640 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:08:46.063527 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:08:46.063955 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:08:46.063957 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:08:46.063958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:08:46.064076 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:08:46.064303 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:08:46.064304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:08:46.064305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:08:46.064586 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:08:46.064669 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 22:08:46.064922 (+ 253us) server_negotiation.cc:299] Negotiation successful
0925 22:08:46.065071 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":52,"threads_started":1}
I20260925 22:08:46.065275 22039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:46.049038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:39851 (local address 127.5.0.65:55895)
0925 22:08:46.049137 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:46.049146 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:08:46.049185 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:08:46.050190 (+ 1005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:08:46.050193 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:08:46.050572 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:08:46.050797 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:46.050804 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:46.051915 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:46.051916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:46.052707 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:08:46.052709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:08:46.053296 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:08:46.053297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:08:46.053407 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:08:46.059297 (+ 5890us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:08:46.059301 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:08:46.061077 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:08:46.063356 (+ 2279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:08:46.063364 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:08:46.063366 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:08:46.063697 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:08:46.064016 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:08:46.064017 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:08:46.064017 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:08:46.064148 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:08:46.064579 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:08:46.064585 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:08:46.064729 (+ 144us) client_negotiation.cc:769] Sending connection context
0925 22:08:46.064834 (+ 105us) client_negotiation.cc:240] Negotiation successful
0925 22:08:46.065131 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":140,"thread_start_us":54,"threads_started":1}
W20260925 22:08:46.066490 21654 leader_election.cc:341] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 69e6a6a4c260409d945be3ac9784d5e8 (127.5.0.66:41847): Not found: Tablet not found: 08f32d3441834064bc997dc036caf356
W20260925 22:08:46.066571 21651 leader_election.cc:341] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 370bc5fa7b7445ac9d0d063337dc9587 (127.5.0.67:39851): Not found: Tablet not found: 08f32d3441834064bc997dc036caf356
I20260925 22:08:46.066649 21651 leader_election.cc:302] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38b225e2ae0f4f8e971038cc071def22; no voters: 370bc5fa7b7445ac9d0d063337dc9587, 69e6a6a4c260409d945be3ac9784d5e8
I20260925 22:08:46.066790 22038 raft_consensus.cc:2726] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 22:08:46.697391 22042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:46.697183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.65:34323)
0925 22:08:46.697315 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:46.697345 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":80,"threads_started":1}
I20260925 22:08:46.793624 22043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:46.793449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.66:47337)
0925 22:08:46.793557 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:46.793590 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":56,"threads_started":1}
I20260925 22:08:46.923777 22044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:46.923574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.67:54881)
0925 22:08:46.923704 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:46.923741 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":76,"threads_started":1}
I20260925 22:08:47.698091 22045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:47.697840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.65:42487)
0925 22:08:47.697996 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:47.698035 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":77,"threads_started":1}
I20260925 22:08:47.794283 22046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:47.794075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.66:54127)
0925 22:08:47.794211 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:47.794246 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":74,"threads_started":1}
I20260925 22:08:47.801759 22047 raft_consensus.cc:482] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:08:47.801833 22047 raft_consensus.cc:504] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e6a6a4c260409d945be3ac9784d5e8" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 41847 } } peers { permanent_uuid: "370bc5fa7b7445ac9d0d063337dc9587" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 39851 } } peers { permanent_uuid: "38b225e2ae0f4f8e971038cc071def22" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37951 } }
I20260925 22:08:47.801985 22047 leader_election.cc:288] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 69e6a6a4c260409d945be3ac9784d5e8 (127.5.0.66:41847), 370bc5fa7b7445ac9d0d063337dc9587 (127.5.0.67:39851)
W20260925 22:08:47.802397 21654 leader_election.cc:341] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 69e6a6a4c260409d945be3ac9784d5e8 (127.5.0.66:41847): Not found: Tablet not found: 08f32d3441834064bc997dc036caf356
W20260925 22:08:47.802505 21651 leader_election.cc:341] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 370bc5fa7b7445ac9d0d063337dc9587 (127.5.0.67:39851): Not found: Tablet not found: 08f32d3441834064bc997dc036caf356
I20260925 22:08:47.802541 21651 leader_election.cc:302] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38b225e2ae0f4f8e971038cc071def22; no voters: 370bc5fa7b7445ac9d0d063337dc9587, 69e6a6a4c260409d945be3ac9784d5e8
I20260925 22:08:47.802630 22047 raft_consensus.cc:2726] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 22:08:47.924549 22048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:47.924291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.67:54907)
0925 22:08:47.924454 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:47.924498 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":82,"threads_started":1}
I20260925 22:08:48.698904 22049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:48.698668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.65:55201)
0925 22:08:48.698809 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:48.698857 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":65,"threads_started":1}
I20260925 22:08:48.794962 22050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:48.794756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.66:35291)
0925 22:08:48.794885 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:48.794923 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":70,"threads_started":1}
I20260925 22:08:48.925366 22051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:48.925137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.67:59413)
0925 22:08:48.925283 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:48.925320 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":77,"threads_started":1}
I20260925 22:08:49.699795 22052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:49.699523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.65:51023)
0925 22:08:49.699701 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:49.699747 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":80,"threads_started":1}
I20260925 22:08:49.795672 22053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:49.795446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.66:50355)
0925 22:08:49.795591 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:49.795632 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":77,"threads_started":1}
I20260925 22:08:49.926103 22054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:49.925873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.67:48943)
0925 22:08:49.926022 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:49.926060 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":76,"threads_started":1}
I20260925 22:08:50.142932 22055 raft_consensus.cc:482] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:08:50.143026 22055 raft_consensus.cc:504] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e6a6a4c260409d945be3ac9784d5e8" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 41847 } } peers { permanent_uuid: "370bc5fa7b7445ac9d0d063337dc9587" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 39851 } } peers { permanent_uuid: "38b225e2ae0f4f8e971038cc071def22" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37951 } }
I20260925 22:08:50.143210 22055 leader_election.cc:288] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 69e6a6a4c260409d945be3ac9784d5e8 (127.5.0.66:41847), 370bc5fa7b7445ac9d0d063337dc9587 (127.5.0.67:39851)
W20260925 22:08:50.143580 21651 leader_election.cc:341] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 370bc5fa7b7445ac9d0d063337dc9587 (127.5.0.67:39851): Not found: Tablet not found: 08f32d3441834064bc997dc036caf356
W20260925 22:08:50.143653 21654 leader_election.cc:341] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 69e6a6a4c260409d945be3ac9784d5e8 (127.5.0.66:41847): Not found: Tablet not found: 08f32d3441834064bc997dc036caf356
I20260925 22:08:50.143743 21654 leader_election.cc:302] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38b225e2ae0f4f8e971038cc071def22; no voters: 370bc5fa7b7445ac9d0d063337dc9587, 69e6a6a4c260409d945be3ac9784d5e8
I20260925 22:08:50.143826 22055 raft_consensus.cc:2726] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 22:08:50.700582 22056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:50.700316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.65:52431)
0925 22:08:50.700504 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:50.700535 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":136,"threads_started":1}
I20260925 22:08:50.796350 22057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:50.796144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.66:37379)
0925 22:08:50.796273 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:50.796306 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":76,"threads_started":1}
I20260925 22:08:50.926908 22058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:50.926677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.67:52843)
0925 22:08:50.926820 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:50.926861 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":67,"threads_started":1}
I20260925 22:08:51.701326 22059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:51.701109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.65:32919)
0925 22:08:51.701241 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:51.701279 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":64,"threads_started":1}
I20260925 22:08:51.797013 22060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:51.796804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.66:33637)
0925 22:08:51.796928 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:51.796960 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":61,"threads_started":1}
I20260925 22:08:51.927645 22061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:51.927431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.67:53085)
0925 22:08:51.927564 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:51.927601 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":66,"threads_started":1}
I20260925 22:08:52.702147 22062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:52.701870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.65:33173)
0925 22:08:52.702061 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:52.702102 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":108,"threads_started":1}
I20260925 22:08:52.797683 22063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:52.797485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.66:41335)
0925 22:08:52.797603 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:52.797639 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
I20260925 22:08:52.891026 22064 raft_consensus.cc:482] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:08:52.891149 22064 raft_consensus.cc:504] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e6a6a4c260409d945be3ac9784d5e8" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 41847 } } peers { permanent_uuid: "370bc5fa7b7445ac9d0d063337dc9587" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 39851 } } peers { permanent_uuid: "38b225e2ae0f4f8e971038cc071def22" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37951 } }
I20260925 22:08:52.891373 22064 leader_election.cc:288] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 69e6a6a4c260409d945be3ac9784d5e8 (127.5.0.66:41847), 370bc5fa7b7445ac9d0d063337dc9587 (127.5.0.67:39851)
W20260925 22:08:52.891832 21654 leader_election.cc:341] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 69e6a6a4c260409d945be3ac9784d5e8 (127.5.0.66:41847): Not found: Tablet not found: 08f32d3441834064bc997dc036caf356
W20260925 22:08:52.891942 21651 leader_election.cc:341] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 370bc5fa7b7445ac9d0d063337dc9587 (127.5.0.67:39851): Not found: Tablet not found: 08f32d3441834064bc997dc036caf356
I20260925 22:08:52.892030 21651 leader_election.cc:302] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38b225e2ae0f4f8e971038cc071def22; no voters: 370bc5fa7b7445ac9d0d063337dc9587, 69e6a6a4c260409d945be3ac9784d5e8
I20260925 22:08:52.892104 22064 raft_consensus.cc:2726] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 22:08:52.928339 22065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:52.928109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.67:43193)
0925 22:08:52.928245 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:52.928282 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":75,"threads_started":1}
I20260925 22:08:53.702987 22066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:53.702747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.65:59731)
0925 22:08:53.702891 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:53.702932 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":64,"threads_started":1}
I20260925 22:08:53.798440 22067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:53.798221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.66:45425)
0925 22:08:53.798350 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:53.798393 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":63,"threads_started":1}
I20260925 22:08:53.929060 22068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:53.928828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.67:50981)
0925 22:08:53.928971 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:53.929012 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":63,"threads_started":1}
I20260925 22:08:54.703859 22069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:54.703631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.65:42057)
0925 22:08:54.703780 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:54.703814 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":67,"threads_started":1}
I20260925 22:08:54.799131 22070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:54.798920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.66:55611)
0925 22:08:54.799045 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:54.799083 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":66,"threads_started":1}
I20260925 22:08:54.929811 22071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:54.929544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.67:55619)
0925 22:08:54.929689 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:54.929731 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":59,"threads_started":1}
I20260925 22:08:55.704691 22072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:55.704475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.65:50489)
0925 22:08:55.704612 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:55.704646 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
I20260925 22:08:55.799815 22073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:55.799578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.66:34743)
0925 22:08:55.799736 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:55.799773 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":76,"threads_started":1}
I20260925 22:08:55.930558 22074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:55.930324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.67:57797)
0925 22:08:55.930476 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:55.930512 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":78,"threads_started":1}
I20260925 22:08:56.705464 22075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:56.705221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.65:33645)
0925 22:08:56.705375 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:56.705416 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":79,"threads_started":1}
I20260925 22:08:56.800521 22076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:56.800301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.66:57877)
0925 22:08:56.800428 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:56.800477 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":58,"threads_started":1}
I20260925 22:08:56.931352 22077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:56.931138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.67:35353)
0925 22:08:56.931272 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:56.931306 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":62,"threads_started":1}
I20260925 22:08:57.706297 22078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:57.706019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.65:47475)
0925 22:08:57.706210 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:57.706251 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":98,"threads_started":1}
I20260925 22:08:57.801275 22079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:57.801045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.66:50671)
0925 22:08:57.801190 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:57.801228 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":68,"threads_started":1}
I20260925 22:08:57.932178 22080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:57.931864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.67:56943)
0925 22:08:57.932093 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:57.932128 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":164,"threads_started":1}
I20260925 22:08:58.706995 22081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:58.706785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.65:40459)
0925 22:08:58.706912 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:58.706948 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":70,"threads_started":1}
I20260925 22:08:58.802103 22082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:58.801813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.66:57429)
0925 22:08:58.802003 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:58.802044 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":84,"threads_started":1}
I20260925 22:08:58.932926 22083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:58.932698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.67:38907)
0925 22:08:58.932838 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:58.932878 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":69,"threads_started":1}
I20260925 22:08:59.104123 22084 raft_consensus.cc:482] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:08:59.104260 22084 raft_consensus.cc:504] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e6a6a4c260409d945be3ac9784d5e8" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 41847 } } peers { permanent_uuid: "370bc5fa7b7445ac9d0d063337dc9587" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 39851 } } peers { permanent_uuid: "38b225e2ae0f4f8e971038cc071def22" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37951 } }
I20260925 22:08:59.104526 22084 leader_election.cc:288] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 69e6a6a4c260409d945be3ac9784d5e8 (127.5.0.66:41847), 370bc5fa7b7445ac9d0d063337dc9587 (127.5.0.67:39851)
W20260925 22:08:59.104899 21651 leader_election.cc:341] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 370bc5fa7b7445ac9d0d063337dc9587 (127.5.0.67:39851): Not found: Tablet not found: 08f32d3441834064bc997dc036caf356
W20260925 22:08:59.105020 21654 leader_election.cc:341] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 69e6a6a4c260409d945be3ac9784d5e8 (127.5.0.66:41847): Not found: Tablet not found: 08f32d3441834064bc997dc036caf356
I20260925 22:08:59.105085 21654 leader_election.cc:302] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38b225e2ae0f4f8e971038cc071def22; no voters: 370bc5fa7b7445ac9d0d063337dc9587, 69e6a6a4c260409d945be3ac9784d5e8
I20260925 22:08:59.105170 22084 raft_consensus.cc:2726] T 08f32d3441834064bc997dc036caf356 P 38b225e2ae0f4f8e971038cc071def22 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 22:08:59.707813 22085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:59.707592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.65:36611)
0925 22:08:59.707728 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:59.707766 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":64,"threads_started":1}
I20260925 22:08:59.802865 22086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:59.802643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.66:59947)
0925 22:08:59.802781 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:59.802815 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":81,"threads_started":1}
I20260925 22:08:59.933796 22087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:08:59.933579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42695 (local address 127.5.0.67:58449)
0925 22:08:59.933713 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 22:08:59.933751 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.126:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":62,"threads_started":1}
I20260925 22:08:59.935719 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 21601
I20260925 22:08:59.940502 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 21768
I20260925 22:08:59.944805 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 21902
2026-09-25T22:08:59Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58443 ms)
[----------] 30 tests from SecurityITest (230965 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:09:00 dist-test-slave-7mqr krb5kdc[22090](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:09:00 dist-test-slave-7mqr krb5kdc[22090](info): set up 2 sockets
Sep 25 22:09:00 dist-test-slave-7mqr krb5kdc[22090](info): commencing operation
krb5kdc: starting...
W20260925 22:09:00.999626 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:09:01 dist-test-slave-7mqr krb5kdc[22090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:09:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:09:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.0.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
I20260925 22:09:01.123490 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:37897
--webserver_interface=127.5.0.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:35301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:37897
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:09:01.205526 22103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:01.205776 22103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:01.205814 22103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:01.206985 22103 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:09:01.207062 22103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:01.207088 22103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:01.207118 22103 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:09:01.207144 22103 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:09:01.208700 22103 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:35301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:37897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:37897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:01.208992 22103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:01.209232 22103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:01.214030 22108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:01.214038 22109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:01.214061 22111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:01.214401 22103 server_base.cc:1034] running on GCE node
I20260925 22:09:01.214663 22103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:01.214956 22103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:01.216116 22103 hybrid_clock.cc:648] HybridClock initialized: now 1790374141216114 us; error 37 us; skew 500 ppm
Sep 25 22:09:01 dist-test-slave-7mqr krb5kdc[22090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374141, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:01.218655 22103 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:09:01.219086 22103 webserver.cc:460] Webserver started at http://127.5.0.126:45159/ using document root <none> and password file <none>
I20260925 22:09:01.219307 22103 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:01.219374 22103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:01.219535 22103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:01.220656 22103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "d1db2cf26e1841678334eda2289f3ddf"
format_stamp: "Formatted at 2026-09-25 22:09:01 on dist-test-slave-7mqr"
server_key: "79d5d3916a2efe414f4cbb3a1a627858"
server_key_iv: "f29f43c8d82c1f2bcf4f5994db563e23"
server_key_version: "encryptionkey@0"
I20260925 22:09:01.221011 22103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "d1db2cf26e1841678334eda2289f3ddf"
format_stamp: "Formatted at 2026-09-25 22:09:01 on dist-test-slave-7mqr"
server_key: "79d5d3916a2efe414f4cbb3a1a627858"
server_key_iv: "f29f43c8d82c1f2bcf4f5994db563e23"
server_key_version: "encryptionkey@0"
I20260925 22:09:01.222301 22103 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:09:01.222995 22118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:01.223145 22103 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:09:01.223249 22103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "d1db2cf26e1841678334eda2289f3ddf"
format_stamp: "Formatted at 2026-09-25 22:09:01 on dist-test-slave-7mqr"
server_key: "79d5d3916a2efe414f4cbb3a1a627858"
server_key_iv: "f29f43c8d82c1f2bcf4f5994db563e23"
server_key_version: "encryptionkey@0"
I20260925 22:09:01.223323 22103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:01.233448 22103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:01.235720 22103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:01.235881 22103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:01.239629 22103 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:37897
I20260925 22:09:01.239667 22170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:37897 every 8 connection(s)
I20260925 22:09:01.240021 22103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:09:01.240892 22171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:01.243124 22171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1db2cf26e1841678334eda2289f3ddf: Bootstrap starting.
I20260925 22:09:01.243716 22171 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1db2cf26e1841678334eda2289f3ddf: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:01.244046 22171 log.cc:824] T 00000000000000000000000000000000 P d1db2cf26e1841678334eda2289f3ddf: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:01.244756 22171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1db2cf26e1841678334eda2289f3ddf: No bootstrap required, opened a new log
I20260925 22:09:01.246295 22171 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1db2cf26e1841678334eda2289f3ddf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1db2cf26e1841678334eda2289f3ddf" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 37897 } }
I20260925 22:09:01.246443 22171 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1db2cf26e1841678334eda2289f3ddf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:01.246490 22171 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1db2cf26e1841678334eda2289f3ddf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1db2cf26e1841678334eda2289f3ddf, State: Initialized, Role: FOLLOWER
I20260925 22:09:01.246649 22171 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1db2cf26e1841678334eda2289f3ddf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1db2cf26e1841678334eda2289f3ddf" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 37897 } }
I20260925 22:09:01.246713 22171 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1db2cf26e1841678334eda2289f3ddf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:09:01.246735 22171 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1db2cf26e1841678334eda2289f3ddf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:09:01.246799 22171 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1db2cf26e1841678334eda2289f3ddf [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:01.247310 22171 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1db2cf26e1841678334eda2289f3ddf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1db2cf26e1841678334eda2289f3ddf" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 37897 } }
I20260925 22:09:01.247426 22171 leader_election.cc:302] T 00000000000000000000000000000000 P d1db2cf26e1841678334eda2289f3ddf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1db2cf26e1841678334eda2289f3ddf; no voters:
I20260925 22:09:01.247609 22171 leader_election.cc:288] T 00000000000000000000000000000000 P d1db2cf26e1841678334eda2289f3ddf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:09:01.247645 22174 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1db2cf26e1841678334eda2289f3ddf [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:09:01.247790 22174 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1db2cf26e1841678334eda2289f3ddf [term 1 LEADER]: Becoming Leader. State: Replica: d1db2cf26e1841678334eda2289f3ddf, State: Running, Role: LEADER
I20260925 22:09:01.247866 22171 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1db2cf26e1841678334eda2289f3ddf [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:09:01.247948 22174 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1db2cf26e1841678334eda2289f3ddf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1db2cf26e1841678334eda2289f3ddf" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 37897 } }
I20260925 22:09:01.248340 22175 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1db2cf26e1841678334eda2289f3ddf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1db2cf26e1841678334eda2289f3ddf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1db2cf26e1841678334eda2289f3ddf" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 37897 } } }
I20260925 22:09:01.248432 22175 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1db2cf26e1841678334eda2289f3ddf [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:01.248487 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 22103
I20260925 22:09:01.248488 22176 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1db2cf26e1841678334eda2289f3ddf [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1db2cf26e1841678334eda2289f3ddf. Latest consensus state: current_term: 1 leader_uuid: "d1db2cf26e1841678334eda2289f3ddf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1db2cf26e1841678334eda2289f3ddf" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 37897 } } }
I20260925 22:09:01.248566 22176 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1db2cf26e1841678334eda2289f3ddf [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:01.248584 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:09:01.248806 22181 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:09:01.248847 5121 external_mini_cluster.cc:1428] Setting key 53fff9bb4004d46b6566911030485272
I20260925 22:09:01.249385 22181 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:09:01.251158 22181 catalog_manager.cc:1347] Generated new cluster ID: c07fcdb5e66144aa8c2035610734d9e7
I20260925 22:09:01.251207 22181 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:09:01 dist-test-slave-7mqr krb5kdc[22090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:09:01.261143 22193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:01.249512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43236 (local address 127.5.0.126:37897)
0925 22:09:01.249793 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:01.249803 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:01.249836 (+ 33us) server_negotiation.cc:407] Connection header received
0925 22:09:01.250096 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:01.250104 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:01.250173 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:01.250331 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:01.251229 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.252651 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.253704 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.254365 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.254477 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:01.257386 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:01.257391 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:01.257391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:01.257403 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:01.259385 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:01.259806 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:01.259809 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:01.259810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:01.259944 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:01.260159 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:01.260160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:01.260160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:01.260468 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:01.260627 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:01.260733 (+ 106us) server_negotiation.cc:299] Negotiation successful
0925 22:09:01.260888 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":68,"threads_started":1}
I20260925 22:09:01.261869 22181 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:09:01.262365 22181 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:09:01.270040 22181 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1db2cf26e1841678334eda2289f3ddf: Generated new TSK 0
I20260925 22:09:01.270216 22181 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.0.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
I20260925 22:09:01.294919 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.65:0
--local_ip_for_outbound_sockets=127.5.0.65
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:37897
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:35301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:09:01.375340 22197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:01.375597 22197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:01.375620 22197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:01.376781 22197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:01.376827 22197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:01.376852 22197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.65
I20260925 22:09:01.378232 22197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:35301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:37897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.65
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:01.378413 22197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:01.378597 22197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:01.383368 22203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:01.383385 22205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:01.383385 22202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:01.383747 22197 server_base.cc:1034] running on GCE node
I20260925 22:09:01.383957 22197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:01.384209 22197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:01.385342 22197 hybrid_clock.cc:648] HybridClock initialized: now 1790374141385332 us; error 26 us; skew 500 ppm
Sep 25 22:09:01 dist-test-slave-7mqr krb5kdc[22090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374141, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:01.387856 22197 init.cc:377] Logged in from keytab as kudu/127.5.0.65@KRBTEST.COM (short username kudu)
I20260925 22:09:01.388360 22197 webserver.cc:460] Webserver started at http://127.5.0.65:35319/ using document root <none> and password file <none>
I20260925 22:09:01.388581 22197 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:01.388624 22197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:01.388764 22197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:01.389840 22197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-0/data/instance:
uuid: "1cad42b78a9d481a902b6f3bbca8dd7e"
format_stamp: "Formatted at 2026-09-25 22:09:01 on dist-test-slave-7mqr"
server_key: "a66127c2dda33333c39bec49d3107af6"
server_key_iv: "d50c1cd9ac0ec970bd7c6dc836000d0d"
server_key_version: "encryptionkey@0"
I20260925 22:09:01.390187 22197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance:
uuid: "1cad42b78a9d481a902b6f3bbca8dd7e"
format_stamp: "Formatted at 2026-09-25 22:09:01 on dist-test-slave-7mqr"
server_key: "a66127c2dda33333c39bec49d3107af6"
server_key_iv: "d50c1cd9ac0ec970bd7c6dc836000d0d"
server_key_version: "encryptionkey@0"
I20260925 22:09:01.391496 22197 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:09:01.392257 22212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:01.392485 22197 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 22:09:01.392587 22197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-0/wal
uuid: "1cad42b78a9d481a902b6f3bbca8dd7e"
format_stamp: "Formatted at 2026-09-25 22:09:01 on dist-test-slave-7mqr"
server_key: "a66127c2dda33333c39bec49d3107af6"
server_key_iv: "d50c1cd9ac0ec970bd7c6dc836000d0d"
server_key_version: "encryptionkey@0"
I20260925 22:09:01.392664 22197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:01.405212 22197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:01.407064 22197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:01.407229 22197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:01.407466 22197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:09:01.407780 22197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:09:01.407835 22197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:01.407871 22197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:09:01.407918 22197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:01.414008 22197 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.65:39063
I20260925 22:09:01.414047 22325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.65:39063 every 8 connection(s)
I20260925 22:09:01.414397 22197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
I20260925 22:09:01.419708 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 22197
I20260925 22:09:01.419833 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance
I20260925 22:09:01.420110 5121 external_mini_cluster.cc:1428] Setting key 8c4b0de8f7891919e9b1c663f93a50dc
Sep 25 22:09:01 dist-test-slave-7mqr krb5kdc[22090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374141, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:09:01.424158 22193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:01.415148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:42351 (local address 127.5.0.126:37897)
0925 22:09:01.415208 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:01.415210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:01.415392 (+ 182us) server_negotiation.cc:407] Connection header received
0925 22:09:01.415852 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:01.415853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:01.415861 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:01.415980 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:01.416861 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.417350 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.418327 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.418528 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.418577 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:01.421086 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:01.421087 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:01.421088 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:01.421094 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:01.422755 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:01.423156 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:01.423157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:01.423157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:01.423219 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:01.423446 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:01.423447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:01.423447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:01.423600 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:01.423669 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:01.423925 (+ 256us) server_negotiation.cc:299] Negotiation successful
0925 22:09:01.424017 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 22:09:01.424252 22328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:01.415121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:37897 (local address 127.5.0.65:42351)
0925 22:09:01.415269 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:01.415309 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:01.415776 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:01.416041 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:01.416044 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:01.416598 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:01.416783 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:01.416788 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.417472 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.417472 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:01.418251 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:01.418253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.418624 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.418625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:01.418725 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:01.419156 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:09:01.419164 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:01.421000 (+ 1836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:01.422851 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:01.422857 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:01.422858 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:01.423078 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:01.423289 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:01.423289 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:01.423289 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:01.423382 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:01.423665 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:01.423669 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:01.423776 (+ 107us) client_negotiation.cc:769] Sending connection context
0925 22:09:01.423857 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 22:09:01.424066 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":80,"threads_started":1}
I20260925 22:09:01.424772 22326 heartbeater.cc:347] Connected to a master server at 127.5.0.126:37897
I20260925 22:09:01.424862 22326 heartbeater.cc:464] Registering TS with master...
I20260925 22:09:01.425050 22326 heartbeater.cc:511] Master 127.5.0.126:37897 requested a full tablet report, sending...
I20260925 22:09:01.425594 22135 ts_manager.cc:194] Registered new tserver with Master: 1cad42b78a9d481a902b6f3bbca8dd7e (127.5.0.65:39063)
I20260925 22:09:01.426255 22135 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.65@KRBTEST.COM'} at 127.5.0.65:42351
WARNING: no policy specified for kudu/127.5.0.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
I20260925 22:09:01.447022 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.66:0
--local_ip_for_outbound_sockets=127.5.0.66
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:37897
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:35301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:09:01.527285 22331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:01.527523 22331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:01.527561 22331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:01.528694 22331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:01.528761 22331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:01.528805 22331 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.66
I20260925 22:09:01.530226 22331 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:35301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:37897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.66
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:01.530486 22331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:01.530714 22331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:01.535547 22339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:01.535576 22337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:01.535547 22336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:01.535820 22331 server_base.cc:1034] running on GCE node
I20260925 22:09:01.536052 22331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:01.536304 22331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:01.537439 22331 hybrid_clock.cc:648] HybridClock initialized: now 1790374141537413 us; error 48 us; skew 500 ppm
Sep 25 22:09:01 dist-test-slave-7mqr krb5kdc[22090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374141, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:01.540058 22331 init.cc:377] Logged in from keytab as kudu/127.5.0.66@KRBTEST.COM (short username kudu)
I20260925 22:09:01.540427 22331 webserver.cc:460] Webserver started at http://127.5.0.66:40863/ using document root <none> and password file <none>
I20260925 22:09:01.540608 22331 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:01.540647 22331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:01.540736 22331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:01.541778 22331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-1/data/instance:
uuid: "dbb5e179ad244602a81784b927d1dd23"
format_stamp: "Formatted at 2026-09-25 22:09:01 on dist-test-slave-7mqr"
server_key: "e17118329cbfcf04183a74db04d9a6f2"
server_key_iv: "50b523dc0c33a35bbbbde8d75f56a8ee"
server_key_version: "encryptionkey@0"
I20260925 22:09:01.542086 22331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance:
uuid: "dbb5e179ad244602a81784b927d1dd23"
format_stamp: "Formatted at 2026-09-25 22:09:01 on dist-test-slave-7mqr"
server_key: "e17118329cbfcf04183a74db04d9a6f2"
server_key_iv: "50b523dc0c33a35bbbbde8d75f56a8ee"
server_key_version: "encryptionkey@0"
I20260925 22:09:01.543326 22331 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:09:01.544203 22346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:01.544375 22331 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:09:01.544499 22331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-1/wal
uuid: "dbb5e179ad244602a81784b927d1dd23"
format_stamp: "Formatted at 2026-09-25 22:09:01 on dist-test-slave-7mqr"
server_key: "e17118329cbfcf04183a74db04d9a6f2"
server_key_iv: "50b523dc0c33a35bbbbde8d75f56a8ee"
server_key_version: "encryptionkey@0"
I20260925 22:09:01.544579 22331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:01.556016 22331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:01.558307 22331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:01.558450 22331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:01.558682 22331 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:09:01.558969 22331 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:09:01.559002 22331 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:01.559051 22331 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:09:01.559080 22331 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:01.565379 22331 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.66:46063
I20260925 22:09:01.565433 22459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.66:46063 every 8 connection(s)
I20260925 22:09:01.565747 22331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
Sep 25 22:09:01 dist-test-slave-7mqr krb5kdc[22090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374141, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:09:01.571753 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 22331
I20260925 22:09:01.571859 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance
I20260925 22:09:01.572167 5121 external_mini_cluster.cc:1428] Setting key cb5b3218b695e52e32105ef12ef38cd8
I20260925 22:09:01.574606 22193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:01.566447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:46781 (local address 127.5.0.126:37897)
0925 22:09:01.566490 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:01.566492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:01.566704 (+ 212us) server_negotiation.cc:407] Connection header received
0925 22:09:01.567148 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:01.567149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:01.567158 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:01.567216 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:01.568148 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.568629 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.569561 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.569745 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.569797 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:01.571750 (+ 1953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:01.571752 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:01.571753 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:01.571760 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:01.573293 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:01.573652 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:01.573653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:01.573654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:01.573713 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:01.573931 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:01.573932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:01.573932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:01.574066 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:01.574134 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:01.574368 (+ 234us) server_negotiation.cc:299] Negotiation successful
0925 22:09:01.574465 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 22:09:01.574779 22462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:01.566410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:37897 (local address 127.5.0.66:46781)
0925 22:09:01.566580 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:01.566620 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:01.567071 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:01.567266 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:01.567270 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:01.567810 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:01.568070 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:01.568076 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.568705 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.568706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:01.569482 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:01.569484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.569814 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.569815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:01.569911 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:01.570369 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:09:01.570377 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:01.571656 (+ 1279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:01.573365 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:01.573370 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:01.573371 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:01.573580 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:01.573775 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:01.573776 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:01.573776 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:01.573866 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:01.574128 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:01.574132 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:01.574236 (+ 104us) client_negotiation.cc:769] Sending connection context
0925 22:09:01.574345 (+ 109us) client_negotiation.cc:240] Negotiation successful
0925 22:09:01.574526 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":107,"threads_started":1}
I20260925 22:09:01.575169 22460 heartbeater.cc:347] Connected to a master server at 127.5.0.126:37897
I20260925 22:09:01.575276 22460 heartbeater.cc:464] Registering TS with master...
I20260925 22:09:01.575443 22460 heartbeater.cc:511] Master 127.5.0.126:37897 requested a full tablet report, sending...
I20260925 22:09:01.575834 22135 ts_manager.cc:194] Registered new tserver with Master: dbb5e179ad244602a81784b927d1dd23 (127.5.0.66:46063)
I20260925 22:09:01.576223 22135 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.66@KRBTEST.COM'} at 127.5.0.66:46781
WARNING: no policy specified for kudu/127.5.0.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
I20260925 22:09:01.598858 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.67:0
--local_ip_for_outbound_sockets=127.5.0.67
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:37897
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:35301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:09:01.678990 22465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:01.679193 22465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:01.679217 22465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:01.680368 22465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:01.680408 22465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:01.680434 22465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.67
I20260925 22:09:01.681813 22465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:35301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:37897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.67
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:01.682004 22465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:01.682184 22465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:01.687208 22473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:01.687250 22471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:01.687390 22465 server_base.cc:1034] running on GCE node
W20260925 22:09:01.687233 22470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:01.687700 22465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:01.687952 22465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:01.689111 22465 hybrid_clock.cc:648] HybridClock initialized: now 1790374141689091 us; error 36 us; skew 500 ppm
Sep 25 22:09:01 dist-test-slave-7mqr krb5kdc[22090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374141, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:01.691707 22465 init.cc:377] Logged in from keytab as kudu/127.5.0.67@KRBTEST.COM (short username kudu)
I20260925 22:09:01.692338 22465 webserver.cc:460] Webserver started at http://127.5.0.67:40051/ using document root <none> and password file <none>
I20260925 22:09:01.692651 22465 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:01.692734 22465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:01.692888 22465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:01.694425 22465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-2/data/instance:
uuid: "251a81ee5a1341e994dc03e500505632"
format_stamp: "Formatted at 2026-09-25 22:09:01 on dist-test-slave-7mqr"
server_key: "a5d3679c165e2cf85b65c0631120de13"
server_key_iv: "12b451c0cfaafe01d081d74fcd052a2d"
server_key_version: "encryptionkey@0"
I20260925 22:09:01.694860 22465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance:
uuid: "251a81ee5a1341e994dc03e500505632"
format_stamp: "Formatted at 2026-09-25 22:09:01 on dist-test-slave-7mqr"
server_key: "a5d3679c165e2cf85b65c0631120de13"
server_key_iv: "12b451c0cfaafe01d081d74fcd052a2d"
server_key_version: "encryptionkey@0"
I20260925 22:09:01.696622 22465 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.004s
I20260925 22:09:01.697391 22480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:01.697563 22465 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:09:01.697649 22465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-2/wal
uuid: "251a81ee5a1341e994dc03e500505632"
format_stamp: "Formatted at 2026-09-25 22:09:01 on dist-test-slave-7mqr"
server_key: "a5d3679c165e2cf85b65c0631120de13"
server_key_iv: "12b451c0cfaafe01d081d74fcd052a2d"
server_key_version: "encryptionkey@0"
I20260925 22:09:01.697712 22465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:01.732672 22465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:01.735080 22465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:01.735191 22465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:01.735381 22465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:09:01.735694 22465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:09:01.735731 22465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:01.735754 22465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:09:01.735800 22465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:01.741614 22465 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.67:44997
I20260925 22:09:01.741701 22593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.67:44997 every 8 connection(s)
I20260925 22:09:01.742002 22465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
I20260925 22:09:01.743609 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 22465
I20260925 22:09:01.743746 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance
I20260925 22:09:01.744000 5121 external_mini_cluster.cc:1428] Setting key 8ff94db63c7406d2714fea493b0af439
Sep 25 22:09:01 dist-test-slave-7mqr krb5kdc[22090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374141, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:09:01.751411 22193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:01.742705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:56715 (local address 127.5.0.126:37897)
0925 22:09:01.742758 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:01.742760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:01.742981 (+ 221us) server_negotiation.cc:407] Connection header received
0925 22:09:01.743476 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:01.743477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:01.743490 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:01.743564 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:01.744702 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.745197 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.746178 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.746379 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.746478 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:01.748863 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:01.748864 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:01.748864 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:01.748870 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:01.750257 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:01.750638 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:01.750639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:01.750639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:01.750688 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:01.750900 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:01.750901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:01.750901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:01.751002 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:01.751064 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:01.751288 (+ 224us) server_negotiation.cc:299] Negotiation successful
0925 22:09:01.751346 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 22:09:01.751634 22596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:01.742695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:37897 (local address 127.5.0.67:56715)
0925 22:09:01.742862 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:01.742898 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:01.743362 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:01.743652 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:01.743655 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:01.744367 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:01.744613 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:01.744619 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.745289 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.745290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:01.746087 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:01.746089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.746487 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.746488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:01.746625 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:01.747108 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:09:01.747115 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:01.748774 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:01.750335 (+ 1561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:01.750340 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:01.750341 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:01.750574 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:01.750746 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:01.750746 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:01.750747 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:01.750847 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:01.751059 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:01.751063 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:01.751174 (+ 111us) client_negotiation.cc:769] Sending connection context
0925 22:09:01.751257 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 22:09:01.751372 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":88,"threads_started":1}
I20260925 22:09:01.752100 22594 heartbeater.cc:347] Connected to a master server at 127.5.0.126:37897
I20260925 22:09:01.752197 22594 heartbeater.cc:464] Registering TS with master...
I20260925 22:09:01.752363 22594 heartbeater.cc:511] Master 127.5.0.126:37897 requested a full tablet report, sending...
I20260925 22:09:01.752772 22135 ts_manager.cc:194] Registered new tserver with Master: 251a81ee5a1341e994dc03e500505632 (127.5.0.67:44997)
I20260925 22:09:01.753149 22135 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.67@KRBTEST.COM'} at 127.5.0.67:56715
I20260925 22:09:01.755523 5121 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:09:01.762176 22193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:01.756914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43252 (local address 127.5.0.126:37897)
0925 22:09:01.756959 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:01.756961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:01.757077 (+ 116us) server_negotiation.cc:407] Connection header received
0925 22:09:01.757128 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:01.757128 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:01.757136 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:01.757180 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:01.757703 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.758149 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.758841 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.759011 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.759058 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:01.760167 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:01.760168 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:01.760168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:01.760173 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:01.761295 (+ 1122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:01.761587 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:01.761588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:01.761588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:01.761633 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:01.761791 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:01.761792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:01.761792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:01.761913 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:01.761982 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:01.762068 (+ 86us) server_negotiation.cc:299] Negotiation successful
0925 22:09:01.762119 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 22:09:01.763623 22135 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43252:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:09:01.765271 22135 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:09:01.771500 22606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:01.767635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46990 (local address 127.5.0.67:44997)
0925 22:09:01.767790 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:01.767794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:01.767803 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:09:01.767842 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:01.767844 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:01.767856 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:01.768001 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:09:01.768268 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.768983 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.770347 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.771108 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.771183 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:01.771244 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:01.771384 (+ 140us) server_negotiation.cc:299] Negotiation successful
0925 22:09:01.771424 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":86,"threads_started":1}
I20260925 22:09:01.771582 22602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:01.767315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:46063 (local address 127.0.0.1:40614)
0925 22:09:01.767617 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:01.767627 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:01.767682 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:01.768078 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:01.768079 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:01.768080 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:09:01.768153 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:01.768154 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.769498 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.769499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:01.770486 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:01.770487 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.771320 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.771321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:01.771395 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:01.771401 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 22:09:01.771430 (+ 29us) client_negotiation.cc:240] Negotiation successful
0925 22:09:01.771469 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":2,"thread_start_us":62,"threads_started":1}
I20260925 22:09:01.771723 22607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:01.767689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40614 (local address 127.5.0.66:46063)
0925 22:09:01.767873 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:01.767876 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:01.767906 (+ 30us) server_negotiation.cc:407] Connection header received
0925 22:09:01.767946 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:01.767948 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:01.767964 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:01.768232 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:09:01.768305 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.769389 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.770551 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.771226 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.771527 (+ 301us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:01.771587 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:01.771611 (+ 24us) server_negotiation.cc:299] Negotiation successful
0925 22:09:01.771646 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":78,"threads_started":1}
I20260925 22:09:01.771482 22605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:01.767503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:44997 (local address 127.0.0.1:46990)
0925 22:09:01.767687 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:01.767697 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:01.767744 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:01.768016 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:01.768019 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:01.768021 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:09:01.768183 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:01.768186 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.769094 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.769095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:01.770267 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:01.770270 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.771197 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.771198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:01.771275 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:01.771292 (+ 17us) client_negotiation.cc:769] Sending connection context
0925 22:09:01.771355 (+ 63us) client_negotiation.cc:240] Negotiation successful
0925 22:09:01.771409 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":56,"threads_started":1}
I20260925 22:09:01.772395 22603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:01.767290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:39063 (local address 127.0.0.1:60766)
0925 22:09:01.767419 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:01.767457 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:01.767515 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:01.768070 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:01.768071 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:01.768073 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:09:01.768153 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:01.768154 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.770568 (+ 2414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.770568 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:01.771244 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:01.771245 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.772167 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.772167 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:01.772231 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:01.772234 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:09:01.772293 (+ 59us) client_negotiation.cc:240] Negotiation successful
0925 22:09:01.772332 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":4,"thread_start_us":71,"threads_started":1}
I20260925 22:09:01.772423 22394 tablet_service.cc:1467] Processing CreateTablet for tablet b6f27c0c7fec4f0b96261eb99373b49a (DEFAULT_TABLE table=test-table [id=3ce34e7c92334311856838080c7be82d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:09:01.772610 22604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:01.767429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60766 (local address 127.5.0.65:39063)
0925 22:09:01.767862 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:01.767867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:01.767878 (+ 11us) server_negotiation.cc:407] Connection header received
0925 22:09:01.767947 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:01.767948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:01.767964 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:01.768232 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:09:01.768306 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.769396 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.771311 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.772074 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.772163 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:01.772216 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:01.772493 (+ 277us) server_negotiation.cc:299] Negotiation successful
0925 22:09:01.772536 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":103,"threads_started":1}
I20260925 22:09:01.772673 22528 tablet_service.cc:1467] Processing CreateTablet for tablet b6f27c0c7fec4f0b96261eb99373b49a (DEFAULT_TABLE table=test-table [id=3ce34e7c92334311856838080c7be82d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:09:01.772854 22394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6f27c0c7fec4f0b96261eb99373b49a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:01.773012 22528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6f27c0c7fec4f0b96261eb99373b49a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:01.773298 22260 tablet_service.cc:1467] Processing CreateTablet for tablet b6f27c0c7fec4f0b96261eb99373b49a (DEFAULT_TABLE table=test-table [id=3ce34e7c92334311856838080c7be82d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:09:01.773587 22260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6f27c0c7fec4f0b96261eb99373b49a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:01.775249 22608 tablet_bootstrap.cc:492] T b6f27c0c7fec4f0b96261eb99373b49a P dbb5e179ad244602a81784b927d1dd23: Bootstrap starting.
I20260925 22:09:01.775955 22608 tablet_bootstrap.cc:654] T b6f27c0c7fec4f0b96261eb99373b49a P dbb5e179ad244602a81784b927d1dd23: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:01.776286 22609 tablet_bootstrap.cc:492] T b6f27c0c7fec4f0b96261eb99373b49a P 251a81ee5a1341e994dc03e500505632: Bootstrap starting.
I20260925 22:09:01.776746 22608 log.cc:824] T b6f27c0c7fec4f0b96261eb99373b49a P dbb5e179ad244602a81784b927d1dd23: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:01.777037 22609 tablet_bootstrap.cc:654] T b6f27c0c7fec4f0b96261eb99373b49a P 251a81ee5a1341e994dc03e500505632: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:01.777243 22610 tablet_bootstrap.cc:492] T b6f27c0c7fec4f0b96261eb99373b49a P 1cad42b78a9d481a902b6f3bbca8dd7e: Bootstrap starting.
I20260925 22:09:01.777640 22608 tablet_bootstrap.cc:492] T b6f27c0c7fec4f0b96261eb99373b49a P dbb5e179ad244602a81784b927d1dd23: No bootstrap required, opened a new log
I20260925 22:09:01.777635 22609 log.cc:824] T b6f27c0c7fec4f0b96261eb99373b49a P 251a81ee5a1341e994dc03e500505632: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:01.777726 22608 ts_tablet_manager.cc:1397] T b6f27c0c7fec4f0b96261eb99373b49a P dbb5e179ad244602a81784b927d1dd23: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 22:09:01.777884 22610 tablet_bootstrap.cc:654] T b6f27c0c7fec4f0b96261eb99373b49a P 1cad42b78a9d481a902b6f3bbca8dd7e: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:01.778292 22610 log.cc:824] T b6f27c0c7fec4f0b96261eb99373b49a P 1cad42b78a9d481a902b6f3bbca8dd7e: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:01.778940 22610 tablet_bootstrap.cc:492] T b6f27c0c7fec4f0b96261eb99373b49a P 1cad42b78a9d481a902b6f3bbca8dd7e: No bootstrap required, opened a new log
I20260925 22:09:01.779023 22610 ts_tablet_manager.cc:1397] T b6f27c0c7fec4f0b96261eb99373b49a P 1cad42b78a9d481a902b6f3bbca8dd7e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:09:01.779027 22609 tablet_bootstrap.cc:492] T b6f27c0c7fec4f0b96261eb99373b49a P 251a81ee5a1341e994dc03e500505632: No bootstrap required, opened a new log
I20260925 22:09:01.779098 22609 ts_tablet_manager.cc:1397] T b6f27c0c7fec4f0b96261eb99373b49a P 251a81ee5a1341e994dc03e500505632: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 22:09:01.779562 22608 raft_consensus.cc:348] T b6f27c0c7fec4f0b96261eb99373b49a P dbb5e179ad244602a81784b927d1dd23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cad42b78a9d481a902b6f3bbca8dd7e" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 39063 } } peers { permanent_uuid: "dbb5e179ad244602a81784b927d1dd23" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 46063 } } peers { permanent_uuid: "251a81ee5a1341e994dc03e500505632" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44997 } }
I20260925 22:09:01.779745 22608 raft_consensus.cc:374] T b6f27c0c7fec4f0b96261eb99373b49a P dbb5e179ad244602a81784b927d1dd23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:01.779780 22608 raft_consensus.cc:729] T b6f27c0c7fec4f0b96261eb99373b49a P dbb5e179ad244602a81784b927d1dd23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbb5e179ad244602a81784b927d1dd23, State: Initialized, Role: FOLLOWER
I20260925 22:09:01.779917 22608 consensus_queue.cc:260] T b6f27c0c7fec4f0b96261eb99373b49a P dbb5e179ad244602a81784b927d1dd23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cad42b78a9d481a902b6f3bbca8dd7e" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 39063 } } peers { permanent_uuid: "dbb5e179ad244602a81784b927d1dd23" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 46063 } } peers { permanent_uuid: "251a81ee5a1341e994dc03e500505632" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44997 } }
I20260925 22:09:01.780205 22608 ts_tablet_manager.cc:1428] T b6f27c0c7fec4f0b96261eb99373b49a P dbb5e179ad244602a81784b927d1dd23: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 22:09:01.780350 22460 heartbeater.cc:503] Master 127.5.0.126:37897 was elected leader, sending a full tablet report...
I20260925 22:09:01.781024 22610 raft_consensus.cc:348] T b6f27c0c7fec4f0b96261eb99373b49a P 1cad42b78a9d481a902b6f3bbca8dd7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cad42b78a9d481a902b6f3bbca8dd7e" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 39063 } } peers { permanent_uuid: "dbb5e179ad244602a81784b927d1dd23" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 46063 } } peers { permanent_uuid: "251a81ee5a1341e994dc03e500505632" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44997 } }
I20260925 22:09:01.781069 22609 raft_consensus.cc:348] T b6f27c0c7fec4f0b96261eb99373b49a P 251a81ee5a1341e994dc03e500505632 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cad42b78a9d481a902b6f3bbca8dd7e" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 39063 } } peers { permanent_uuid: "dbb5e179ad244602a81784b927d1dd23" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 46063 } } peers { permanent_uuid: "251a81ee5a1341e994dc03e500505632" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44997 } }
I20260925 22:09:01.781201 22610 raft_consensus.cc:374] T b6f27c0c7fec4f0b96261eb99373b49a P 1cad42b78a9d481a902b6f3bbca8dd7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:01.781240 22610 raft_consensus.cc:729] T b6f27c0c7fec4f0b96261eb99373b49a P 1cad42b78a9d481a902b6f3bbca8dd7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cad42b78a9d481a902b6f3bbca8dd7e, State: Initialized, Role: FOLLOWER
I20260925 22:09:01.781240 22609 raft_consensus.cc:374] T b6f27c0c7fec4f0b96261eb99373b49a P 251a81ee5a1341e994dc03e500505632 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:01.781279 22609 raft_consensus.cc:729] T b6f27c0c7fec4f0b96261eb99373b49a P 251a81ee5a1341e994dc03e500505632 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 251a81ee5a1341e994dc03e500505632, State: Initialized, Role: FOLLOWER
I20260925 22:09:01.781349 22610 consensus_queue.cc:260] T b6f27c0c7fec4f0b96261eb99373b49a P 1cad42b78a9d481a902b6f3bbca8dd7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cad42b78a9d481a902b6f3bbca8dd7e" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 39063 } } peers { permanent_uuid: "dbb5e179ad244602a81784b927d1dd23" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 46063 } } peers { permanent_uuid: "251a81ee5a1341e994dc03e500505632" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44997 } }
I20260925 22:09:01.781387 22609 consensus_queue.cc:260] T b6f27c0c7fec4f0b96261eb99373b49a P 251a81ee5a1341e994dc03e500505632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cad42b78a9d481a902b6f3bbca8dd7e" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 39063 } } peers { permanent_uuid: "dbb5e179ad244602a81784b927d1dd23" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 46063 } } peers { permanent_uuid: "251a81ee5a1341e994dc03e500505632" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44997 } }
I20260925 22:09:01.781639 22610 ts_tablet_manager.cc:1428] T b6f27c0c7fec4f0b96261eb99373b49a P 1cad42b78a9d481a902b6f3bbca8dd7e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 22:09:01.781679 22326 heartbeater.cc:503] Master 127.5.0.126:37897 was elected leader, sending a full tablet report...
I20260925 22:09:01.781670 22609 ts_tablet_manager.cc:1428] T b6f27c0c7fec4f0b96261eb99373b49a P 251a81ee5a1341e994dc03e500505632: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 22:09:01.781679 22594 heartbeater.cc:503] Master 127.5.0.126:37897 was elected leader, sending a full tablet report...
W20260925 22:09:01.816298 22461 tablet.cc:2369] T b6f27c0c7fec4f0b96261eb99373b49a P dbb5e179ad244602a81784b927d1dd23: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:09:01.855906 22615 raft_consensus.cc:482] T b6f27c0c7fec4f0b96261eb99373b49a P 1cad42b78a9d481a902b6f3bbca8dd7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:09:01.856030 22615 raft_consensus.cc:504] T b6f27c0c7fec4f0b96261eb99373b49a P 1cad42b78a9d481a902b6f3bbca8dd7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cad42b78a9d481a902b6f3bbca8dd7e" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 39063 } } peers { permanent_uuid: "dbb5e179ad244602a81784b927d1dd23" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 46063 } } peers { permanent_uuid: "251a81ee5a1341e994dc03e500505632" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44997 } }
I20260925 22:09:01.856377 22615 leader_election.cc:288] T b6f27c0c7fec4f0b96261eb99373b49a P 1cad42b78a9d481a902b6f3bbca8dd7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbb5e179ad244602a81784b927d1dd23 (127.5.0.66:46063), 251a81ee5a1341e994dc03e500505632 (127.5.0.67:44997)
I20260925 22:09:01.859596 22606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:01.856872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:43259 (local address 127.5.0.67:44997)
0925 22:09:01.856928 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:01.856930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:01.857043 (+ 113us) server_negotiation.cc:407] Connection header received
0925 22:09:01.857091 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:01.857093 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:01.857103 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:01.857159 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:09:01.857311 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.857760 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.858835 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.859195 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.859250 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:01.859258 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:01.859474 (+ 216us) server_negotiation.cc:299] Negotiation successful
0925 22:09:01.859514 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 22:09:01.859609 22607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:01.856724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:39667 (local address 127.5.0.66:46063)
0925 22:09:01.856780 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:01.856782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:01.856792 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:09:01.856822 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:01.856823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:01.856835 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:01.856886 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:09:01.857145 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.857752 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.858765 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.859215 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.859315 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:01.859326 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:01.859479 (+ 153us) server_negotiation.cc:299] Negotiation successful
0925 22:09:01.859520 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 22:09:01.859627 22617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:01.856743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:44997 (local address 127.5.0.65:43259)
0925 22:09:01.856946 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:01.856965 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:01.857038 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:01.857172 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:01.857173 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:01.857175 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:09:01.857242 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:01.857244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.857871 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.857871 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:01.858743 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:01.858746 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.859287 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.859288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:01.859370 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:01.859375 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:09:01.859469 (+ 94us) client_negotiation.cc:240] Negotiation successful
0925 22:09:01.859517 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":1,"thread_start_us":142,"threads_started":1}
I20260925 22:09:01.859632 22328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:01.856613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:46063 (local address 127.5.0.65:39667)
0925 22:09:01.856681 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:01.856695 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:01.856758 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:01.856947 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:01.856948 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:01.856949 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:09:01.857044 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:01.857046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.857848 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.857849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:01.858663 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:01.858666 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.859334 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.859335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:01.859396 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:01.859399 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:09:01.859469 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 22:09:01.859513 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":60}
I20260925 22:09:01.860083 22548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6f27c0c7fec4f0b96261eb99373b49a" candidate_uuid: "1cad42b78a9d481a902b6f3bbca8dd7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "251a81ee5a1341e994dc03e500505632" is_pre_election: true
I20260925 22:09:01.860265 22548 raft_consensus.cc:2454] T b6f27c0c7fec4f0b96261eb99373b49a P 251a81ee5a1341e994dc03e500505632 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cad42b78a9d481a902b6f3bbca8dd7e in term 0.
I20260925 22:09:01.860236 22414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6f27c0c7fec4f0b96261eb99373b49a" candidate_uuid: "1cad42b78a9d481a902b6f3bbca8dd7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbb5e179ad244602a81784b927d1dd23" is_pre_election: true
I20260925 22:09:01.860400 22414 raft_consensus.cc:2454] T b6f27c0c7fec4f0b96261eb99373b49a P dbb5e179ad244602a81784b927d1dd23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cad42b78a9d481a902b6f3bbca8dd7e in term 0.
I20260925 22:09:01.860527 22214 leader_election.cc:302] T b6f27c0c7fec4f0b96261eb99373b49a P 1cad42b78a9d481a902b6f3bbca8dd7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cad42b78a9d481a902b6f3bbca8dd7e, 251a81ee5a1341e994dc03e500505632; no voters:
I20260925 22:09:01.860669 22615 raft_consensus.cc:2781] T b6f27c0c7fec4f0b96261eb99373b49a P 1cad42b78a9d481a902b6f3bbca8dd7e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:09:01.860730 22615 raft_consensus.cc:482] T b6f27c0c7fec4f0b96261eb99373b49a P 1cad42b78a9d481a902b6f3bbca8dd7e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:09:01.860756 22615 raft_consensus.cc:3037] T b6f27c0c7fec4f0b96261eb99373b49a P 1cad42b78a9d481a902b6f3bbca8dd7e [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:01.861426 22615 raft_consensus.cc:504] T b6f27c0c7fec4f0b96261eb99373b49a P 1cad42b78a9d481a902b6f3bbca8dd7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cad42b78a9d481a902b6f3bbca8dd7e" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 39063 } } peers { permanent_uuid: "dbb5e179ad244602a81784b927d1dd23" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 46063 } } peers { permanent_uuid: "251a81ee5a1341e994dc03e500505632" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44997 } }
I20260925 22:09:01.861562 22615 leader_election.cc:288] T b6f27c0c7fec4f0b96261eb99373b49a P 1cad42b78a9d481a902b6f3bbca8dd7e [CANDIDATE]: Term 1 election: Requested vote from peers dbb5e179ad244602a81784b927d1dd23 (127.5.0.66:46063), 251a81ee5a1341e994dc03e500505632 (127.5.0.67:44997)
I20260925 22:09:01.861744 22414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6f27c0c7fec4f0b96261eb99373b49a" candidate_uuid: "1cad42b78a9d481a902b6f3bbca8dd7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbb5e179ad244602a81784b927d1dd23"
I20260925 22:09:01.861744 22548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6f27c0c7fec4f0b96261eb99373b49a" candidate_uuid: "1cad42b78a9d481a902b6f3bbca8dd7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "251a81ee5a1341e994dc03e500505632"
I20260925 22:09:01.861822 22414 raft_consensus.cc:3037] T b6f27c0c7fec4f0b96261eb99373b49a P dbb5e179ad244602a81784b927d1dd23 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:01.861845 22548 raft_consensus.cc:3037] T b6f27c0c7fec4f0b96261eb99373b49a P 251a81ee5a1341e994dc03e500505632 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:01.862561 22414 raft_consensus.cc:2454] T b6f27c0c7fec4f0b96261eb99373b49a P dbb5e179ad244602a81784b927d1dd23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cad42b78a9d481a902b6f3bbca8dd7e in term 1.
I20260925 22:09:01.862728 22214 leader_election.cc:302] T b6f27c0c7fec4f0b96261eb99373b49a P 1cad42b78a9d481a902b6f3bbca8dd7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cad42b78a9d481a902b6f3bbca8dd7e, dbb5e179ad244602a81784b927d1dd23; no voters:
I20260925 22:09:01.862831 22615 raft_consensus.cc:2781] T b6f27c0c7fec4f0b96261eb99373b49a P 1cad42b78a9d481a902b6f3bbca8dd7e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:09:01.862864 22548 raft_consensus.cc:2454] T b6f27c0c7fec4f0b96261eb99373b49a P 251a81ee5a1341e994dc03e500505632 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cad42b78a9d481a902b6f3bbca8dd7e in term 1.
I20260925 22:09:01.862998 22615 raft_consensus.cc:686] T b6f27c0c7fec4f0b96261eb99373b49a P 1cad42b78a9d481a902b6f3bbca8dd7e [term 1 LEADER]: Becoming Leader. State: Replica: 1cad42b78a9d481a902b6f3bbca8dd7e, State: Running, Role: LEADER
I20260925 22:09:01.863094 22615 consensus_queue.cc:237] T b6f27c0c7fec4f0b96261eb99373b49a P 1cad42b78a9d481a902b6f3bbca8dd7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cad42b78a9d481a902b6f3bbca8dd7e" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 39063 } } peers { permanent_uuid: "dbb5e179ad244602a81784b927d1dd23" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 46063 } } peers { permanent_uuid: "251a81ee5a1341e994dc03e500505632" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44997 } }
I20260925 22:09:01.864063 22134 catalog_manager.cc:5495] T b6f27c0c7fec4f0b96261eb99373b49a P 1cad42b78a9d481a902b6f3bbca8dd7e reported cstate change: term changed from 0 to 1, leader changed from <none> to 1cad42b78a9d481a902b6f3bbca8dd7e (127.5.0.65). New cstate: current_term: 1 leader_uuid: "1cad42b78a9d481a902b6f3bbca8dd7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cad42b78a9d481a902b6f3bbca8dd7e" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 39063 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbb5e179ad244602a81784b927d1dd23" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 46063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "251a81ee5a1341e994dc03e500505632" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44997 } health_report { overall_health: UNKNOWN } } }
I20260925 22:09:01.891084 22604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:01.888718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60776 (local address 127.5.0.65:39063)
0925 22:09:01.888783 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:01.888787 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:01.888799 (+ 12us) server_negotiation.cc:407] Connection header received
0925 22:09:01.888838 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:01.888839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:01.888860 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:01.888930 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:09:01.889056 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.889523 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.890278 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:01.890495 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:01.890563 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:01.890737 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:09:01.890864 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:09:01.890947 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:01.890969 (+ 22us) server_negotiation.cc:299] Negotiation successful
0925 22:09:01.890991 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 22:09:01.893110 22414 raft_consensus.cc:1261] T b6f27c0c7fec4f0b96261eb99373b49a P dbb5e179ad244602a81784b927d1dd23 [term 1 FOLLOWER]: Refusing update from remote peer 1cad42b78a9d481a902b6f3bbca8dd7e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:09:01.893128 22548 raft_consensus.cc:1261] T b6f27c0c7fec4f0b96261eb99373b49a P 251a81ee5a1341e994dc03e500505632 [term 1 FOLLOWER]: Refusing update from remote peer 1cad42b78a9d481a902b6f3bbca8dd7e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:09:01.893497 22615 consensus_queue.cc:1035] T b6f27c0c7fec4f0b96261eb99373b49a P 1cad42b78a9d481a902b6f3bbca8dd7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbb5e179ad244602a81784b927d1dd23" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 46063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:09:01.893673 22618 consensus_queue.cc:1035] T b6f27c0c7fec4f0b96261eb99373b49a P 1cad42b78a9d481a902b6f3bbca8dd7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "251a81ee5a1341e994dc03e500505632" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 44997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:09:01.896126 22621 mvcc.cc:205] Tried to move back new op lower bound from 7333372485191233536 to 7333372485071863808. Current Snapshot: MvccSnapshot[applied={T|T < 7333372485191233536}]
I20260925 22:09:01.897055 22623 mvcc.cc:205] Tried to move back new op lower bound from 7333372485191233536 to 7333372485071863808. Current Snapshot: MvccSnapshot[applied={T|T < 7333372485191233536 or (T in {7333372485191233536})}]
I20260925 22:09:01.898095 22622 mvcc.cc:205] Tried to move back new op lower bound from 7333372485191233536 to 7333372485071863808. Current Snapshot: MvccSnapshot[applied={T|T < 7333372485191233536 or (T in {7333372485191233536})}]
I20260925 22:09:01.898430 22133 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:09:01.898535 22133 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:09:01.899269 22133 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d1db2cf26e1841678334eda2289f3ddf: Sending DeleteTablet for 3 replicas of tablet b6f27c0c7fec4f0b96261eb99373b49a
I20260925 22:09:01.899569 22260 tablet_service.cc:1514] Processing DeleteTablet for tablet b6f27c0c7fec4f0b96261eb99373b49a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:09:01 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:60766
I20260925 22:09:01.899638 22394 tablet_service.cc:1514] Processing DeleteTablet for tablet b6f27c0c7fec4f0b96261eb99373b49a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:09:01 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:40614
I20260925 22:09:01.899811 22630 tablet_replica.cc:330] T b6f27c0c7fec4f0b96261eb99373b49a P 1cad42b78a9d481a902b6f3bbca8dd7e: stopping tablet replica
I20260925 22:09:01.899926 22630 raft_consensus.cc:2229] T b6f27c0c7fec4f0b96261eb99373b49a P 1cad42b78a9d481a902b6f3bbca8dd7e [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:09:01.899938 22528 tablet_service.cc:1514] Processing DeleteTablet for tablet b6f27c0c7fec4f0b96261eb99373b49a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:09:01 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:46990
I20260925 22:09:01.900040 22630 raft_consensus.cc:2258] T b6f27c0c7fec4f0b96261eb99373b49a P 1cad42b78a9d481a902b6f3bbca8dd7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:09:01.900060 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 22197
I20260925 22:09:01.900635 22631 tablet_replica.cc:330] T b6f27c0c7fec4f0b96261eb99373b49a P dbb5e179ad244602a81784b927d1dd23: stopping tablet replica
I20260925 22:09:01.900730 22631 raft_consensus.cc:2229] T b6f27c0c7fec4f0b96261eb99373b49a P dbb5e179ad244602a81784b927d1dd23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:09:01.900825 22631 raft_consensus.cc:2258] T b6f27c0c7fec4f0b96261eb99373b49a P dbb5e179ad244602a81784b927d1dd23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:09:01.901281 22631 ts_tablet_manager.cc:1905] T b6f27c0c7fec4f0b96261eb99373b49a P dbb5e179ad244602a81784b927d1dd23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:09:01.903379 22631 ts_tablet_manager.cc:1918] T b6f27c0c7fec4f0b96261eb99373b49a P dbb5e179ad244602a81784b927d1dd23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:09:01.903460 22631 log.cc:1196] T b6f27c0c7fec4f0b96261eb99373b49a P dbb5e179ad244602a81784b927d1dd23: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-1/wal/wals/b6f27c0c7fec4f0b96261eb99373b49a
I20260925 22:09:01.903730 22631 ts_tablet_manager.cc:1939] T b6f27c0c7fec4f0b96261eb99373b49a P dbb5e179ad244602a81784b927d1dd23: Deleting consensus metadata
I20260925 22:09:01.904036 22632 tablet_replica.cc:330] T b6f27c0c7fec4f0b96261eb99373b49a P 251a81ee5a1341e994dc03e500505632: stopping tablet replica
I20260925 22:09:01.904157 22632 raft_consensus.cc:2229] T b6f27c0c7fec4f0b96261eb99373b49a P 251a81ee5a1341e994dc03e500505632 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:09:01.904181 22120 catalog_manager.cc:4841] TS dbb5e179ad244602a81784b927d1dd23 (127.5.0.66:46063): tablet b6f27c0c7fec4f0b96261eb99373b49a (table test-table [id=3ce34e7c92334311856838080c7be82d]) successfully deleted
I20260925 22:09:01.904269 22632 raft_consensus.cc:2258] T b6f27c0c7fec4f0b96261eb99373b49a P 251a81ee5a1341e994dc03e500505632 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:09:01.904618 22632 ts_tablet_manager.cc:1905] T b6f27c0c7fec4f0b96261eb99373b49a P 251a81ee5a1341e994dc03e500505632: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 22:09:01.906474 22119 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.0.65:39063 (error 108)
I20260925 22:09:01.906682 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 22331
I20260925 22:09:01.907285 22603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:01.906790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:39063 (local address 127.0.0.1:60784)
0925 22:09:01.907211 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:01.907255 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.65:39063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400}
W20260925 22:09:01.907434 22119 catalog_manager.cc:4584] TS 1cad42b78a9d481a902b6f3bbca8dd7e (127.5.0.65:39063): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b6f27c0c7fec4f0b96261eb99373b49a: Network error: Client connection negotiation failed: client connection to 127.5.0.65:39063: connect: Connection refused (error 111)
I20260925 22:09:01.908004 22632 ts_tablet_manager.cc:1918] T b6f27c0c7fec4f0b96261eb99373b49a P 251a81ee5a1341e994dc03e500505632: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:09:01.908097 22632 log.cc:1196] T b6f27c0c7fec4f0b96261eb99373b49a P 251a81ee5a1341e994dc03e500505632: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373908987421-5121-0/minicluster-data/ts-2/wal/wals/b6f27c0c7fec4f0b96261eb99373b49a
I20260925 22:09:01.908354 22632 ts_tablet_manager.cc:1939] T b6f27c0c7fec4f0b96261eb99373b49a P 251a81ee5a1341e994dc03e500505632: Deleting consensus metadata
I20260925 22:09:01.908722 22120 catalog_manager.cc:4841] TS 251a81ee5a1341e994dc03e500505632 (127.5.0.67:44997): tablet b6f27c0c7fec4f0b96261eb99373b49a (table test-table [id=3ce34e7c92334311856838080c7be82d]) successfully deleted
I20260925 22:09:01.912168 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 22465
I20260925 22:09:01.916882 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 22103
2026-09-25T22:09:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1971 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-25T22:09:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:09:01Z Disabled control of system clock
I20260925 22:09:01.944132 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:33689
--webserver_interface=127.5.0.126
--webserver_port=0
--builtin_ntp_servers=127.5.0.84:43145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:33689
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 22:09:02.025954 22637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:02.026188 22637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:02.026211 22637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:02.027387 22637 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:09:02.027432 22637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:02.027446 22637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:02.027458 22637 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:09:02.027470 22637 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:09:02.028854 22637 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:43145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:33689
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:33689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:02.029050 22637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:02.029290 22637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:02.034027 22642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:02.034145 22645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:02.034027 22643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:02.034819 22637 server_base.cc:1034] running on GCE node
I20260925 22:09:02.035107 22637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:02.035449 22637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:02.036633 22637 hybrid_clock.cc:648] HybridClock initialized: now 1790374142036610 us; error 35 us; skew 500 ppm
I20260925 22:09:02.038215 22637 webserver.cc:460] Webserver started at http://127.5.0.126:35305/ using document root <none> and password file <none>
I20260925 22:09:02.038502 22637 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:02.038574 22637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:02.038712 22637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:02.040102 22637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "dd3989f71292422abd5dc3466c214a7a"
format_stamp: "Formatted at 2026-09-25 22:09:02 on dist-test-slave-7mqr"
server_key: "da9ffe38464c889389659ef01a603064"
server_key_iv: "98a69ce63ef8fd41c2dcf3ab17c657bb"
server_key_version: "encryptionkey@0"
I20260925 22:09:02.040575 22637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "dd3989f71292422abd5dc3466c214a7a"
format_stamp: "Formatted at 2026-09-25 22:09:02 on dist-test-slave-7mqr"
server_key: "da9ffe38464c889389659ef01a603064"
server_key_iv: "98a69ce63ef8fd41c2dcf3ab17c657bb"
server_key_version: "encryptionkey@0"
I20260925 22:09:02.042218 22637 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:09:02.043085 22651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:02.043267 22637 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:09:02.043361 22637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "dd3989f71292422abd5dc3466c214a7a"
format_stamp: "Formatted at 2026-09-25 22:09:02 on dist-test-slave-7mqr"
server_key: "da9ffe38464c889389659ef01a603064"
server_key_iv: "98a69ce63ef8fd41c2dcf3ab17c657bb"
server_key_version: "encryptionkey@0"
I20260925 22:09:02.043483 22637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:02.090761 22637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:02.091130 22637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:02.091295 22637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:02.095386 22637 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:33689
I20260925 22:09:02.095440 22703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:33689 every 8 connection(s)
I20260925 22:09:02.095821 22637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:09:02.096822 22704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:02.098872 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 22637
I20260925 22:09:02.099028 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:09:02.099192 22704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd3989f71292422abd5dc3466c214a7a: Bootstrap starting.
I20260925 22:09:02.099226 5121 external_mini_cluster.cc:1428] Setting key f0b5d4126c66a2b9a34fb4da304a1a4e
I20260925 22:09:02.099890 22704 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd3989f71292422abd5dc3466c214a7a: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:02.100240 22704 log.cc:824] T 00000000000000000000000000000000 P dd3989f71292422abd5dc3466c214a7a: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:02.100996 22704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd3989f71292422abd5dc3466c214a7a: No bootstrap required, opened a new log
I20260925 22:09:02.103179 22704 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd3989f71292422abd5dc3466c214a7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd3989f71292422abd5dc3466c214a7a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 33689 } }
I20260925 22:09:02.103315 22704 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd3989f71292422abd5dc3466c214a7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:02.103338 22704 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd3989f71292422abd5dc3466c214a7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd3989f71292422abd5dc3466c214a7a, State: Initialized, Role: FOLLOWER
I20260925 22:09:02.103467 22704 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd3989f71292422abd5dc3466c214a7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd3989f71292422abd5dc3466c214a7a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 33689 } }
I20260925 22:09:02.103542 22704 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd3989f71292422abd5dc3466c214a7a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:09:02.103564 22704 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd3989f71292422abd5dc3466c214a7a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:09:02.103631 22704 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd3989f71292422abd5dc3466c214a7a [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:02.104420 22704 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd3989f71292422abd5dc3466c214a7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd3989f71292422abd5dc3466c214a7a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 33689 } }
I20260925 22:09:02.104523 22704 leader_election.cc:302] T 00000000000000000000000000000000 P dd3989f71292422abd5dc3466c214a7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd3989f71292422abd5dc3466c214a7a; no voters:
I20260925 22:09:02.104708 22704 leader_election.cc:288] T 00000000000000000000000000000000 P dd3989f71292422abd5dc3466c214a7a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:09:02.104753 22709 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd3989f71292422abd5dc3466c214a7a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:09:02.104914 22709 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd3989f71292422abd5dc3466c214a7a [term 1 LEADER]: Becoming Leader. State: Replica: dd3989f71292422abd5dc3466c214a7a, State: Running, Role: LEADER
I20260925 22:09:02.105018 22709 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd3989f71292422abd5dc3466c214a7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd3989f71292422abd5dc3466c214a7a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 33689 } }
I20260925 22:09:02.104991 22704 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd3989f71292422abd5dc3466c214a7a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:09:02.105381 22707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:02.099773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45982 (local address 127.5.0.126:33689)
0925 22:09:02.100007 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:02.100016 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:02.100051 (+ 35us) server_negotiation.cc:407] Connection header received
0925 22:09:02.100551 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:02.100558 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:02.100619 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:02.100746 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:02.100964 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.101879 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.103065 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.103627 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.103750 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:02.104408 (+ 658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:02.104411 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:02.104411 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:02.104650 (+ 239us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:02.104693 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:02.104709 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:02.105054 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:02.105091 (+ 37us) server_negotiation.cc:299] Negotiation successful
0925 22:09:02.105172 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":95,"threads_started":1}
I20260925 22:09:02.106458 22711 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd3989f71292422abd5dc3466c214a7a [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd3989f71292422abd5dc3466c214a7a. Latest consensus state: current_term: 1 leader_uuid: "dd3989f71292422abd5dc3466c214a7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd3989f71292422abd5dc3466c214a7a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 33689 } } }
I20260925 22:09:02.106561 22711 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd3989f71292422abd5dc3466c214a7a [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:02.106794 22710 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd3989f71292422abd5dc3466c214a7a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd3989f71292422abd5dc3466c214a7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd3989f71292422abd5dc3466c214a7a" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 33689 } } }
I20260925 22:09:02.106922 22710 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd3989f71292422abd5dc3466c214a7a [sys.catalog]: This master's current role is: LEADER
W20260925 22:09:02.107064 22724 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dd3989f71292422abd5dc3466c214a7a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:09:02.107131 22724 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:09:02.107195 22725 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:09:02.107335 22725 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:09:02.108731 22725 catalog_manager.cc:1347] Generated new cluster ID: 82f13f53a704471182507920aa9ad401
I20260925 22:09:02.108783 22725 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:09:02.114252 22725 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:09:02.114953 22725 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:09:02.120942 22725 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd3989f71292422abd5dc3466c214a7a: Generated new TSK 0
I20260925 22:09:02.121110 22725 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:09:02.125156 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.65:0
--local_ip_for_outbound_sockets=127.5.0.65
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:33689
--builtin_ntp_servers=127.5.0.84:43145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 22:09:02.206614 22728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:02.206821 22728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:02.206843 22728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:02.208029 22728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:02.208068 22728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:02.208093 22728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.65
I20260925 22:09:02.209432 22728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:43145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:33689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.65
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:02.209681 22728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:02.210007 22728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:02.214869 22733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:02.214875 22736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:02.214883 22734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:02.215127 22728 server_base.cc:1034] running on GCE node
I20260925 22:09:02.215445 22728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:02.215685 22728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:02.216848 22728 hybrid_clock.cc:648] HybridClock initialized: now 1790374142216819 us; error 48 us; skew 500 ppm
I20260925 22:09:02.218183 22728 webserver.cc:460] Webserver started at http://127.5.0.65:40975/ using document root <none> and password file <none>
I20260925 22:09:02.218437 22728 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:02.218494 22728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:02.218643 22728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:02.220011 22728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-0/data/instance:
uuid: "3a1e349d9f014f0fa2831b96bcb9ee66"
format_stamp: "Formatted at 2026-09-25 22:09:02 on dist-test-slave-7mqr"
server_key: "8c8c9ef0568a04f643b560e833a2aae7"
server_key_iv: "c01e1a7e2acdeb5f7b2b5f59c56df303"
server_key_version: "encryptionkey@0"
I20260925 22:09:02.220395 22728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance:
uuid: "3a1e349d9f014f0fa2831b96bcb9ee66"
format_stamp: "Formatted at 2026-09-25 22:09:02 on dist-test-slave-7mqr"
server_key: "8c8c9ef0568a04f643b560e833a2aae7"
server_key_iv: "c01e1a7e2acdeb5f7b2b5f59c56df303"
server_key_version: "encryptionkey@0"
I20260925 22:09:02.222077 22728 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:09:02.222894 22742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:02.223104 22728 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:09:02.223193 22728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-0/wal
uuid: "3a1e349d9f014f0fa2831b96bcb9ee66"
format_stamp: "Formatted at 2026-09-25 22:09:02 on dist-test-slave-7mqr"
server_key: "8c8c9ef0568a04f643b560e833a2aae7"
server_key_iv: "c01e1a7e2acdeb5f7b2b5f59c56df303"
server_key_version: "encryptionkey@0"
I20260925 22:09:02.223261 22728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:02.246615 22728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:02.246903 22728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:02.247059 22728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:02.247318 22728 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:09:02.247680 22728 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:09:02.247751 22728 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:02.247787 22728 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:09:02.247812 22728 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:02.253875 22728 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.65:37067
I20260925 22:09:02.253965 22855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.65:37067 every 8 connection(s)
I20260925 22:09:02.254292 22728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
I20260925 22:09:02.259524 22707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:02.255069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:51793 (local address 127.5.0.126:33689)
0925 22:09:02.255120 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:02.255122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:02.255344 (+ 222us) server_negotiation.cc:407] Connection header received
0925 22:09:02.255833 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:02.255834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:02.255844 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:02.256072 (+ 228us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:02.256744 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.257250 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.258191 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.258379 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.258442 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:02.259259 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:02.259260 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:02.259260 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:02.259264 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:02.259279 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:02.259283 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:02.259327 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:02.259407 (+ 80us) server_negotiation.cc:299] Negotiation successful
0925 22:09:02.259433 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 22:09:02.259677 22858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:02.255024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:33689 (local address 127.5.0.65:51793)
0925 22:09:02.255207 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:02.255248 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:02.255728 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:02.256141 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:02.256145 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:02.256153 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:02.256655 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:02.256659 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.257331 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.257332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:02.258107 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:02.258109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.258464 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.258464 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:02.258577 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:02.259129 (+ 552us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:09:02.259129 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:02.259157 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:09:02.259179 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:02.259338 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:02.259342 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:02.259344 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 22:09:02.259397 (+ 53us) client_negotiation.cc:240] Negotiation successful
0925 22:09:02.259475 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
I20260925 22:09:02.260242 22856 heartbeater.cc:347] Connected to a master server at 127.5.0.126:33689
I20260925 22:09:02.260335 22856 heartbeater.cc:464] Registering TS with master...
I20260925 22:09:02.260391 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 22728
I20260925 22:09:02.260496 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance
I20260925 22:09:02.260541 22856 heartbeater.cc:511] Master 127.5.0.126:33689 requested a full tablet report, sending...
I20260925 22:09:02.260697 5121 external_mini_cluster.cc:1428] Setting key a6a6b4da7ca02edc699f4ac2198880cd
I20260925 22:09:02.261145 22668 ts_manager.cc:194] Registered new tserver with Master: 3a1e349d9f014f0fa2831b96bcb9ee66 (127.5.0.65:37067)
I20260925 22:09:02.261806 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.66:0
--local_ip_for_outbound_sockets=127.5.0.66
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:33689
--builtin_ntp_servers=127.5.0.84:43145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 22:09:02.261971 22668 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.0.65:51793
W20260925 22:09:02.342301 22859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:02.342504 22859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:02.342532 22859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:02.343637 22859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:02.343669 22859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:02.343693 22859 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.66
I20260925 22:09:02.345021 22859 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:43145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:33689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.66
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:02.345206 22859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:02.345431 22859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:02.350407 22865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:02.350492 22864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:02.350534 22859 server_base.cc:1034] running on GCE node
W20260925 22:09:02.350414 22867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:02.350870 22859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:02.351082 22859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:02.352214 22859 hybrid_clock.cc:648] HybridClock initialized: now 1790374142352204 us; error 26 us; skew 500 ppm
I20260925 22:09:02.353417 22859 webserver.cc:460] Webserver started at http://127.5.0.66:39531/ using document root <none> and password file <none>
I20260925 22:09:02.353636 22859 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:02.353696 22859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:02.353826 22859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:02.354846 22859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-1/data/instance:
uuid: "a40ee039ec9e4a028b77e1ac82ff52b5"
format_stamp: "Formatted at 2026-09-25 22:09:02 on dist-test-slave-7mqr"
server_key: "f72d1487766afcaed425c8d291637829"
server_key_iv: "422f149c9f3bfe7ed05a16451f7b4546"
server_key_version: "encryptionkey@0"
I20260925 22:09:02.355223 22859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance:
uuid: "a40ee039ec9e4a028b77e1ac82ff52b5"
format_stamp: "Formatted at 2026-09-25 22:09:02 on dist-test-slave-7mqr"
server_key: "f72d1487766afcaed425c8d291637829"
server_key_iv: "422f149c9f3bfe7ed05a16451f7b4546"
server_key_version: "encryptionkey@0"
I20260925 22:09:02.356590 22859 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:09:02.357354 22873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:02.357523 22859 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:09:02.357620 22859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-1/wal
uuid: "a40ee039ec9e4a028b77e1ac82ff52b5"
format_stamp: "Formatted at 2026-09-25 22:09:02 on dist-test-slave-7mqr"
server_key: "f72d1487766afcaed425c8d291637829"
server_key_iv: "422f149c9f3bfe7ed05a16451f7b4546"
server_key_version: "encryptionkey@0"
I20260925 22:09:02.357702 22859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:02.375515 22859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:02.375852 22859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:02.376047 22859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:02.376314 22859 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:09:02.376665 22859 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:09:02.376720 22859 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:02.376775 22859 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:09:02.376801 22859 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:02.382830 22859 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.66:36095
I20260925 22:09:02.382879 22986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.66:36095 every 8 connection(s)
I20260925 22:09:02.383229 22859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
I20260925 22:09:02.386482 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 22859
I20260925 22:09:02.386612 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance
I20260925 22:09:02.386811 5121 external_mini_cluster.cc:1428] Setting key dd073ead5c40d684fe0fe2f8bb495203
I20260925 22:09:02.387750 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.67:0
--local_ip_for_outbound_sockets=127.5.0.67
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:33689
--builtin_ntp_servers=127.5.0.84:43145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 22:09:02.388955 22707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:02.384013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:35349 (local address 127.5.0.126:33689)
0925 22:09:02.384097 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:02.384099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:02.384310 (+ 211us) server_negotiation.cc:407] Connection header received
0925 22:09:02.384873 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:02.384874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:02.384887 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:02.384952 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:02.385669 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.386179 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.387458 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.387645 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.387719 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:02.388702 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:02.388704 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:02.388704 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:02.388709 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:02.388724 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:02.388728 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:02.388773 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:02.388848 (+ 75us) server_negotiation.cc:299] Negotiation successful
0925 22:09:02.388878 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 22:09:02.389196 22989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:02.383978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:33689 (local address 127.5.0.66:35349)
0925 22:09:02.384183 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:02.384218 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:02.384755 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:02.385047 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:02.385055 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:02.385064 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:02.385572 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:02.385576 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.386279 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.386280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:02.387302 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:02.387305 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.387727 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.387728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:02.387935 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:02.388552 (+ 617us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:09:02.388553 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:02.388588 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:09:02.388610 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:02.388783 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:02.388787 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:02.388790 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:09:02.388854 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 22:09:02.388954 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":81,"threads_started":1}
I20260925 22:09:02.389848 22987 heartbeater.cc:347] Connected to a master server at 127.5.0.126:33689
I20260925 22:09:02.389930 22987 heartbeater.cc:464] Registering TS with master...
I20260925 22:09:02.390129 22987 heartbeater.cc:511] Master 127.5.0.126:33689 requested a full tablet report, sending...
I20260925 22:09:02.390524 22668 ts_manager.cc:194] Registered new tserver with Master: a40ee039ec9e4a028b77e1ac82ff52b5 (127.5.0.66:36095)
I20260925 22:09:02.390899 22668 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.0.66:35349
W20260925 22:09:02.469521 22990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:02.469754 22990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:02.469791 22990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:02.470913 22990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:02.470974 22990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:02.471016 22990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.67
I20260925 22:09:02.472405 22990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:43145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:33689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.67
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:02.472659 22990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:02.472939 22990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:02.477934 22998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:02.477864 22996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:02.477864 22995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:02.478174 22990 server_base.cc:1034] running on GCE node
I20260925 22:09:02.478430 22990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:02.478755 22990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:02.480067 22990 hybrid_clock.cc:648] HybridClock initialized: now 1790374142480045 us; error 38 us; skew 500 ppm
I20260925 22:09:02.481331 22990 webserver.cc:460] Webserver started at http://127.5.0.67:46815/ using document root <none> and password file <none>
I20260925 22:09:02.481631 22990 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:02.481704 22990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:02.481843 22990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:02.482892 22990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-2/data/instance:
uuid: "9fdf5d1978a54d9fab8f80da1b992446"
format_stamp: "Formatted at 2026-09-25 22:09:02 on dist-test-slave-7mqr"
server_key: "48db50514598857f50ed3ff79fae6489"
server_key_iv: "e0fe9cafc042790868e5005acc0f6fc2"
server_key_version: "encryptionkey@0"
I20260925 22:09:02.483278 22990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance:
uuid: "9fdf5d1978a54d9fab8f80da1b992446"
format_stamp: "Formatted at 2026-09-25 22:09:02 on dist-test-slave-7mqr"
server_key: "48db50514598857f50ed3ff79fae6489"
server_key_iv: "e0fe9cafc042790868e5005acc0f6fc2"
server_key_version: "encryptionkey@0"
I20260925 22:09:02.484639 22990 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:09:02.485368 23004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:02.485538 22990 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:09:02.485635 22990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-2/wal
uuid: "9fdf5d1978a54d9fab8f80da1b992446"
format_stamp: "Formatted at 2026-09-25 22:09:02 on dist-test-slave-7mqr"
server_key: "48db50514598857f50ed3ff79fae6489"
server_key_iv: "e0fe9cafc042790868e5005acc0f6fc2"
server_key_version: "encryptionkey@0"
I20260925 22:09:02.485714 22990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:02.502648 22990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:02.502969 22990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:02.503126 22990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:02.503357 22990 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:09:02.503692 22990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:09:02.503747 22990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:02.503783 22990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:09:02.503813 22990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:02.509824 22990 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.67:33029
I20260925 22:09:02.509878 23117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.67:33029 every 8 connection(s)
I20260925 22:09:02.510229 22990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
I20260925 22:09:02.513511 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 22990
I20260925 22:09:02.513641 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance
I20260925 22:09:02.513844 5121 external_mini_cluster.cc:1428] Setting key 62f17a7b6fb2af557ac715ddb5844ea3
I20260925 22:09:02.515623 22707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:02.510975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:50983 (local address 127.5.0.126:33689)
0925 22:09:02.511066 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:02.511068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:02.511276 (+ 208us) server_negotiation.cc:407] Connection header received
0925 22:09:02.511761 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:02.511762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:02.511774 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:02.511829 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:02.512568 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.513093 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.514299 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.514583 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.514636 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:02.515417 (+ 781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:02.515418 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:02.515418 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:02.515422 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:02.515435 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:02.515439 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:02.515485 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:02.515540 (+ 55us) server_negotiation.cc:299] Negotiation successful
0925 22:09:02.515560 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 22:09:02.515882 23120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:02.510928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:33689 (local address 127.5.0.67:50983)
0925 22:09:02.511146 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:02.511180 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:02.511670 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:02.511940 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:02.511944 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:02.511955 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:02.512481 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:02.512486 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.513186 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.513187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:02.514182 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:02.514184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.514662 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.514664 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:02.514798 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:02.515307 (+ 509us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:09:02.515307 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:02.515335 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:09:02.515357 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:02.515488 (+ 131us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:02.515491 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:02.515493 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 22:09:02.515542 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 22:09:02.515623 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":106,"threads_started":1}
I20260925 22:09:02.516347 23118 heartbeater.cc:347] Connected to a master server at 127.5.0.126:33689
I20260925 22:09:02.516441 23118 heartbeater.cc:464] Registering TS with master...
I20260925 22:09:02.516645 23118 heartbeater.cc:511] Master 127.5.0.126:33689 requested a full tablet report, sending...
I20260925 22:09:02.517012 22668 ts_manager.cc:194] Registered new tserver with Master: 9fdf5d1978a54d9fab8f80da1b992446 (127.5.0.67:33029)
I20260925 22:09:02.517328 22668 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.0.67:50983
I20260925 22:09:02.525640 5121 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:09:02.529942 22707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:02.527011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52382 (local address 127.5.0.126:33689)
0925 22:09:02.527069 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:02.527071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:02.527220 (+ 149us) server_negotiation.cc:407] Connection header received
0925 22:09:02.527274 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:02.527275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:02.527283 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:02.527322 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:02.527474 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.528110 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.528872 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.529048 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.529097 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:02.529711 (+ 614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:02.529712 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:02.529713 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:02.529716 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:02.529730 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:02.529734 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:02.529780 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:02.529816 (+ 36us) server_negotiation.cc:299] Negotiation successful
0925 22:09:02.529833 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 22:09:02.531648 22668 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52382:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:09:02.533262 22668 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:09:02.539427 23128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:02.535654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33964 (local address 127.5.0.65:37067)
0925 22:09:02.535960 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:02.535965 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:02.535976 (+ 11us) server_negotiation.cc:407] Connection header received
0925 22:09:02.536106 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:02.536109 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:02.536126 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:02.536196 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:09:02.536428 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.537151 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.538460 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.539051 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.539123 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:02.539206 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:02.539304 (+ 98us) server_negotiation.cc:299] Negotiation successful
0925 22:09:02.539342 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":88,"threads_started":1}
I20260925 22:09:02.539436 23127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:02.535529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:37067 (local address 127.0.0.1:33964)
0925 22:09:02.535847 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:02.535910 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:02.535980 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:02.536269 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:02.536272 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:02.536274 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:09:02.536337 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:02.536339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.537240 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.537241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:02.538383 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:02.538386 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.539134 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.539135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:02.539209 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:02.539222 (+ 13us) client_negotiation.cc:769] Sending connection context
0925 22:09:02.539291 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 22:09:02.539340 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":60,"threads_started":1}
I20260925 22:09:02.540160 23130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:02.535725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:33029 (local address 127.0.0.1:59536)
0925 22:09:02.536036 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:02.536045 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:02.536094 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:02.536330 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:02.536331 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:02.536332 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:09:02.536371 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:02.536372 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.537695 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.537696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:02.538744 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:02.538745 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.539933 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.539934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:02.539995 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:02.540000 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:09:02.540050 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 22:09:02.540090 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":74,"threads_started":1}
I20260925 22:09:02.540174 23131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:02.535800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59536 (local address 127.5.0.67:33029)
0925 22:09:02.536039 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:02.536043 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:02.536154 (+ 111us) server_negotiation.cc:407] Connection header received
0925 22:09:02.536194 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:02.536196 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:02.536210 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:02.536275 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:09:02.536443 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.537608 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.538831 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.539825 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.539938 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:02.540022 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:02.540059 (+ 37us) server_negotiation.cc:299] Negotiation successful
0925 22:09:02.540093 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":77,"threads_started":1}
I20260925 22:09:02.540361 22790 tablet_service.cc:1467] Processing CreateTablet for tablet 81139f9dbf23419a809572772dd05720 (DEFAULT_TABLE table=test-table [id=a2b714982ef04406b8aefb85f125c906]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:09:02.540752 22790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81139f9dbf23419a809572772dd05720. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:02.540833 23052 tablet_service.cc:1467] Processing CreateTablet for tablet 81139f9dbf23419a809572772dd05720 (DEFAULT_TABLE table=test-table [id=a2b714982ef04406b8aefb85f125c906]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:09:02.541172 23052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81139f9dbf23419a809572772dd05720. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:02.543267 23132 tablet_bootstrap.cc:492] T 81139f9dbf23419a809572772dd05720 P 3a1e349d9f014f0fa2831b96bcb9ee66: Bootstrap starting.
I20260925 22:09:02.543756 23133 tablet_bootstrap.cc:492] T 81139f9dbf23419a809572772dd05720 P 9fdf5d1978a54d9fab8f80da1b992446: Bootstrap starting.
I20260925 22:09:02.544108 23129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:02.535654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35852 (local address 127.5.0.66:36095)
0925 22:09:02.535853 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:02.535858 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:02.536098 (+ 240us) server_negotiation.cc:407] Connection header received
0925 22:09:02.536189 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:02.536191 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:02.536208 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:02.536263 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:09:02.536500 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.537605 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.539454 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.543567 (+ 4113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.543630 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:02.543716 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:02.543960 (+ 244us) server_negotiation.cc:299] Negotiation successful
0925 22:09:02.544013 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":93,"threads_started":1}
I20260925 22:09:02.544250 23132 tablet_bootstrap.cc:654] T 81139f9dbf23419a809572772dd05720 P 3a1e349d9f014f0fa2831b96bcb9ee66: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:02.544335 23126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:02.535541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:36095 (local address 127.0.0.1:35852)
0925 22:09:02.535853 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:02.535910 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:02.535991 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:02.536388 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:02.536389 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:02.536391 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:09:02.536437 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:02.536439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.538486 (+ 2047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.538487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:02.539369 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:02.539371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.543753 (+ 4382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.543753 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:02.543822 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:02.543826 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:09:02.543880 (+ 54us) client_negotiation.cc:240] Negotiation successful
0925 22:09:02.544267 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":79,"threads_started":1}
I20260925 22:09:02.544523 23133 tablet_bootstrap.cc:654] T 81139f9dbf23419a809572772dd05720 P 9fdf5d1978a54d9fab8f80da1b992446: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:02.544567 23132 log.cc:824] T 81139f9dbf23419a809572772dd05720 P 3a1e349d9f014f0fa2831b96bcb9ee66: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:02.544848 23133 log.cc:824] T 81139f9dbf23419a809572772dd05720 P 9fdf5d1978a54d9fab8f80da1b992446: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:02.545086 22921 tablet_service.cc:1467] Processing CreateTablet for tablet 81139f9dbf23419a809572772dd05720 (DEFAULT_TABLE table=test-table [id=a2b714982ef04406b8aefb85f125c906]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:09:02.545342 23132 tablet_bootstrap.cc:492] T 81139f9dbf23419a809572772dd05720 P 3a1e349d9f014f0fa2831b96bcb9ee66: No bootstrap required, opened a new log
I20260925 22:09:02.545413 23132 ts_tablet_manager.cc:1397] T 81139f9dbf23419a809572772dd05720 P 3a1e349d9f014f0fa2831b96bcb9ee66: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 22:09:02.545425 22921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81139f9dbf23419a809572772dd05720. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:02.545625 23133 tablet_bootstrap.cc:492] T 81139f9dbf23419a809572772dd05720 P 9fdf5d1978a54d9fab8f80da1b992446: No bootstrap required, opened a new log
I20260925 22:09:02.545688 23133 ts_tablet_manager.cc:1397] T 81139f9dbf23419a809572772dd05720 P 9fdf5d1978a54d9fab8f80da1b992446: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:09:02.547747 23132 raft_consensus.cc:348] T 81139f9dbf23419a809572772dd05720 P 3a1e349d9f014f0fa2831b96bcb9ee66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a40ee039ec9e4a028b77e1ac82ff52b5" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 36095 } } peers { permanent_uuid: "3a1e349d9f014f0fa2831b96bcb9ee66" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37067 } } peers { permanent_uuid: "9fdf5d1978a54d9fab8f80da1b992446" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33029 } }
I20260925 22:09:02.547927 23132 raft_consensus.cc:374] T 81139f9dbf23419a809572772dd05720 P 3a1e349d9f014f0fa2831b96bcb9ee66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:02.547979 23132 raft_consensus.cc:729] T 81139f9dbf23419a809572772dd05720 P 3a1e349d9f014f0fa2831b96bcb9ee66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a1e349d9f014f0fa2831b96bcb9ee66, State: Initialized, Role: FOLLOWER
I20260925 22:09:02.548105 23132 consensus_queue.cc:260] T 81139f9dbf23419a809572772dd05720 P 3a1e349d9f014f0fa2831b96bcb9ee66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a40ee039ec9e4a028b77e1ac82ff52b5" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 36095 } } peers { permanent_uuid: "3a1e349d9f014f0fa2831b96bcb9ee66" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37067 } } peers { permanent_uuid: "9fdf5d1978a54d9fab8f80da1b992446" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33029 } }
I20260925 22:09:02.548409 22856 heartbeater.cc:503] Master 127.5.0.126:33689 was elected leader, sending a full tablet report...
I20260925 22:09:02.548395 23132 ts_tablet_manager.cc:1428] T 81139f9dbf23419a809572772dd05720 P 3a1e349d9f014f0fa2831b96bcb9ee66: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260925 22:09:02.548552 23133 raft_consensus.cc:348] T 81139f9dbf23419a809572772dd05720 P 9fdf5d1978a54d9fab8f80da1b992446 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a40ee039ec9e4a028b77e1ac82ff52b5" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 36095 } } peers { permanent_uuid: "3a1e349d9f014f0fa2831b96bcb9ee66" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37067 } } peers { permanent_uuid: "9fdf5d1978a54d9fab8f80da1b992446" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33029 } }
I20260925 22:09:02.548707 23133 raft_consensus.cc:374] T 81139f9dbf23419a809572772dd05720 P 9fdf5d1978a54d9fab8f80da1b992446 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:02.548744 23133 raft_consensus.cc:729] T 81139f9dbf23419a809572772dd05720 P 9fdf5d1978a54d9fab8f80da1b992446 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fdf5d1978a54d9fab8f80da1b992446, State: Initialized, Role: FOLLOWER
I20260925 22:09:02.548868 23133 consensus_queue.cc:260] T 81139f9dbf23419a809572772dd05720 P 9fdf5d1978a54d9fab8f80da1b992446 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a40ee039ec9e4a028b77e1ac82ff52b5" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 36095 } } peers { permanent_uuid: "3a1e349d9f014f0fa2831b96bcb9ee66" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37067 } } peers { permanent_uuid: "9fdf5d1978a54d9fab8f80da1b992446" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33029 } }
I20260925 22:09:02.549114 23133 ts_tablet_manager.cc:1428] T 81139f9dbf23419a809572772dd05720 P 9fdf5d1978a54d9fab8f80da1b992446: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 22:09:02.549185 23118 heartbeater.cc:503] Master 127.5.0.126:33689 was elected leader, sending a full tablet report...
I20260925 22:09:02.549425 23137 tablet_bootstrap.cc:492] T 81139f9dbf23419a809572772dd05720 P a40ee039ec9e4a028b77e1ac82ff52b5: Bootstrap starting.
I20260925 22:09:02.550108 23137 tablet_bootstrap.cc:654] T 81139f9dbf23419a809572772dd05720 P a40ee039ec9e4a028b77e1ac82ff52b5: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:02.550352 23137 log.cc:824] T 81139f9dbf23419a809572772dd05720 P a40ee039ec9e4a028b77e1ac82ff52b5: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:02.550899 23137 tablet_bootstrap.cc:492] T 81139f9dbf23419a809572772dd05720 P a40ee039ec9e4a028b77e1ac82ff52b5: No bootstrap required, opened a new log
I20260925 22:09:02.550948 23137 ts_tablet_manager.cc:1397] T 81139f9dbf23419a809572772dd05720 P a40ee039ec9e4a028b77e1ac82ff52b5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:09:02.552814 23137 raft_consensus.cc:348] T 81139f9dbf23419a809572772dd05720 P a40ee039ec9e4a028b77e1ac82ff52b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a40ee039ec9e4a028b77e1ac82ff52b5" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 36095 } } peers { permanent_uuid: "3a1e349d9f014f0fa2831b96bcb9ee66" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37067 } } peers { permanent_uuid: "9fdf5d1978a54d9fab8f80da1b992446" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33029 } }
I20260925 22:09:02.552937 23137 raft_consensus.cc:374] T 81139f9dbf23419a809572772dd05720 P a40ee039ec9e4a028b77e1ac82ff52b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:02.552958 23137 raft_consensus.cc:729] T 81139f9dbf23419a809572772dd05720 P a40ee039ec9e4a028b77e1ac82ff52b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a40ee039ec9e4a028b77e1ac82ff52b5, State: Initialized, Role: FOLLOWER
I20260925 22:09:02.553074 23137 consensus_queue.cc:260] T 81139f9dbf23419a809572772dd05720 P a40ee039ec9e4a028b77e1ac82ff52b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a40ee039ec9e4a028b77e1ac82ff52b5" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 36095 } } peers { permanent_uuid: "3a1e349d9f014f0fa2831b96bcb9ee66" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37067 } } peers { permanent_uuid: "9fdf5d1978a54d9fab8f80da1b992446" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33029 } }
I20260925 22:09:02.553289 23137 ts_tablet_manager.cc:1428] T 81139f9dbf23419a809572772dd05720 P a40ee039ec9e4a028b77e1ac82ff52b5: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 22:09:02.553364 22987 heartbeater.cc:503] Master 127.5.0.126:33689 was elected leader, sending a full tablet report...
I20260925 22:09:02.558352 23138 raft_consensus.cc:482] T 81139f9dbf23419a809572772dd05720 P 9fdf5d1978a54d9fab8f80da1b992446 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:09:02.558429 23138 raft_consensus.cc:504] T 81139f9dbf23419a809572772dd05720 P 9fdf5d1978a54d9fab8f80da1b992446 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a40ee039ec9e4a028b77e1ac82ff52b5" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 36095 } } peers { permanent_uuid: "3a1e349d9f014f0fa2831b96bcb9ee66" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37067 } } peers { permanent_uuid: "9fdf5d1978a54d9fab8f80da1b992446" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33029 } }
I20260925 22:09:02.558732 23138 leader_election.cc:288] T 81139f9dbf23419a809572772dd05720 P 9fdf5d1978a54d9fab8f80da1b992446 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a40ee039ec9e4a028b77e1ac82ff52b5 (127.5.0.66:36095), 3a1e349d9f014f0fa2831b96bcb9ee66 (127.5.0.65:37067)
I20260925 22:09:02.562500 23129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:02.558936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:59009 (local address 127.5.0.66:36095)
0925 22:09:02.558976 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:02.558978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:02.558987 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:09:02.559074 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:02.559075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:02.559101 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:02.559166 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:09:02.559348 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.560132 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.561595 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.562144 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.562222 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:02.562231 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:02.562393 (+ 162us) server_negotiation.cc:299] Negotiation successful
0925 22:09:02.562431 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:09:02.562498 23128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:02.558934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:59101 (local address 127.5.0.65:37067)
0925 22:09:02.558975 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:02.558977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:02.559106 (+ 129us) server_negotiation.cc:407] Connection header received
0925 22:09:02.559150 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:02.559151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:02.559163 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:02.559213 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:09:02.559348 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.560132 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.561595 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.562162 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.562222 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:02.562232 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:02.562393 (+ 161us) server_negotiation.cc:299] Negotiation successful
0925 22:09:02.562433 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 22:09:02.562506 23120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:02.558881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:36095 (local address 127.5.0.67:59009)
0925 22:09:02.558933 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:02.558944 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:02.558983 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:02.559185 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:02.559186 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:02.559187 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:09:02.559274 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:02.559276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.560211 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.560211 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:02.561485 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:02.561488 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.562236 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.562237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:02.562313 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:02.562317 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:09:02.562374 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 22:09:02.562418 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":59}
I20260925 22:09:02.562757 23141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:02.558890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:37067 (local address 127.5.0.67:59101)
0925 22:09:02.559031 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:02.559041 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:02.559101 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:02.559224 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:02.559224 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:02.559226 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:09:02.559277 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:02.559278 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.560211 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.560211 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:02.561487 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:02.561488 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.562237 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.562237 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:02.562313 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:02.562318 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:09:02.562374 (+ 56us) client_negotiation.cc:240] Negotiation successful
0925 22:09:02.562416 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":43,"thread_start_us":111,"threads_started":1}
I20260925 22:09:02.562808 22941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81139f9dbf23419a809572772dd05720" candidate_uuid: "9fdf5d1978a54d9fab8f80da1b992446" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a40ee039ec9e4a028b77e1ac82ff52b5" is_pre_election: true
I20260925 22:09:02.562956 22941 raft_consensus.cc:2454] T 81139f9dbf23419a809572772dd05720 P a40ee039ec9e4a028b77e1ac82ff52b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9fdf5d1978a54d9fab8f80da1b992446 in term 0.
I20260925 22:09:02.563108 22810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81139f9dbf23419a809572772dd05720" candidate_uuid: "9fdf5d1978a54d9fab8f80da1b992446" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a1e349d9f014f0fa2831b96bcb9ee66" is_pre_election: true
I20260925 22:09:02.563197 23005 leader_election.cc:302] T 81139f9dbf23419a809572772dd05720 P 9fdf5d1978a54d9fab8f80da1b992446 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9fdf5d1978a54d9fab8f80da1b992446, a40ee039ec9e4a028b77e1ac82ff52b5; no voters:
I20260925 22:09:02.563270 22810 raft_consensus.cc:2454] T 81139f9dbf23419a809572772dd05720 P 3a1e349d9f014f0fa2831b96bcb9ee66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9fdf5d1978a54d9fab8f80da1b992446 in term 0.
I20260925 22:09:02.563366 23138 raft_consensus.cc:2781] T 81139f9dbf23419a809572772dd05720 P 9fdf5d1978a54d9fab8f80da1b992446 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:09:02.563429 23138 raft_consensus.cc:482] T 81139f9dbf23419a809572772dd05720 P 9fdf5d1978a54d9fab8f80da1b992446 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:09:02.563457 23138 raft_consensus.cc:3037] T 81139f9dbf23419a809572772dd05720 P 9fdf5d1978a54d9fab8f80da1b992446 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:02.564184 23138 raft_consensus.cc:504] T 81139f9dbf23419a809572772dd05720 P 9fdf5d1978a54d9fab8f80da1b992446 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a40ee039ec9e4a028b77e1ac82ff52b5" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 36095 } } peers { permanent_uuid: "3a1e349d9f014f0fa2831b96bcb9ee66" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37067 } } peers { permanent_uuid: "9fdf5d1978a54d9fab8f80da1b992446" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33029 } }
I20260925 22:09:02.564321 23138 leader_election.cc:288] T 81139f9dbf23419a809572772dd05720 P 9fdf5d1978a54d9fab8f80da1b992446 [CANDIDATE]: Term 1 election: Requested vote from peers a40ee039ec9e4a028b77e1ac82ff52b5 (127.5.0.66:36095), 3a1e349d9f014f0fa2831b96bcb9ee66 (127.5.0.65:37067)
I20260925 22:09:02.564491 22810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81139f9dbf23419a809572772dd05720" candidate_uuid: "9fdf5d1978a54d9fab8f80da1b992446" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a1e349d9f014f0fa2831b96bcb9ee66"
I20260925 22:09:02.564512 22941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81139f9dbf23419a809572772dd05720" candidate_uuid: "9fdf5d1978a54d9fab8f80da1b992446" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a40ee039ec9e4a028b77e1ac82ff52b5"
I20260925 22:09:02.564594 22941 raft_consensus.cc:3037] T 81139f9dbf23419a809572772dd05720 P a40ee039ec9e4a028b77e1ac82ff52b5 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:02.564574 22810 raft_consensus.cc:3037] T 81139f9dbf23419a809572772dd05720 P 3a1e349d9f014f0fa2831b96bcb9ee66 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:02.565266 22941 raft_consensus.cc:2454] T 81139f9dbf23419a809572772dd05720 P a40ee039ec9e4a028b77e1ac82ff52b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fdf5d1978a54d9fab8f80da1b992446 in term 1.
I20260925 22:09:02.565330 22810 raft_consensus.cc:2454] T 81139f9dbf23419a809572772dd05720 P 3a1e349d9f014f0fa2831b96bcb9ee66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fdf5d1978a54d9fab8f80da1b992446 in term 1.
I20260925 22:09:02.565465 23005 leader_election.cc:302] T 81139f9dbf23419a809572772dd05720 P 9fdf5d1978a54d9fab8f80da1b992446 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9fdf5d1978a54d9fab8f80da1b992446, a40ee039ec9e4a028b77e1ac82ff52b5; no voters:
I20260925 22:09:02.565564 23138 raft_consensus.cc:2781] T 81139f9dbf23419a809572772dd05720 P 9fdf5d1978a54d9fab8f80da1b992446 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:09:02.565744 23138 raft_consensus.cc:686] T 81139f9dbf23419a809572772dd05720 P 9fdf5d1978a54d9fab8f80da1b992446 [term 1 LEADER]: Becoming Leader. State: Replica: 9fdf5d1978a54d9fab8f80da1b992446, State: Running, Role: LEADER
I20260925 22:09:02.565843 23138 consensus_queue.cc:237] T 81139f9dbf23419a809572772dd05720 P 9fdf5d1978a54d9fab8f80da1b992446 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a40ee039ec9e4a028b77e1ac82ff52b5" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 36095 } } peers { permanent_uuid: "3a1e349d9f014f0fa2831b96bcb9ee66" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37067 } } peers { permanent_uuid: "9fdf5d1978a54d9fab8f80da1b992446" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33029 } }
I20260925 22:09:02.566679 22667 catalog_manager.cc:5495] T 81139f9dbf23419a809572772dd05720 P 9fdf5d1978a54d9fab8f80da1b992446 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9fdf5d1978a54d9fab8f80da1b992446 (127.5.0.67). New cstate: current_term: 1 leader_uuid: "9fdf5d1978a54d9fab8f80da1b992446" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a40ee039ec9e4a028b77e1ac82ff52b5" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 36095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a1e349d9f014f0fa2831b96bcb9ee66" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9fdf5d1978a54d9fab8f80da1b992446" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 33029 } health_report { overall_health: HEALTHY } } }
I20260925 22:09:02.574645 23131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:02.572611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59538 (local address 127.5.0.67:33029)
0925 22:09:02.572655 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:02.572656 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:02.572665 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:09:02.572734 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:02.572734 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:02.572748 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:02.572806 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:09:02.572898 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.573334 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.574023 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:02.574202 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:02.574256 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:02.574377 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:09:02.574461 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:09:02.574537 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:02.574565 (+ 28us) server_negotiation.cc:299] Negotiation successful
0925 22:09:02.574580 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 22:09:02.576318 22941 raft_consensus.cc:1261] T 81139f9dbf23419a809572772dd05720 P a40ee039ec9e4a028b77e1ac82ff52b5 [term 1 FOLLOWER]: Refusing update from remote peer 9fdf5d1978a54d9fab8f80da1b992446: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:09:02.576362 22810 raft_consensus.cc:1261] T 81139f9dbf23419a809572772dd05720 P 3a1e349d9f014f0fa2831b96bcb9ee66 [term 1 FOLLOWER]: Refusing update from remote peer 9fdf5d1978a54d9fab8f80da1b992446: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:09:02.576696 23142 consensus_queue.cc:1035] T 81139f9dbf23419a809572772dd05720 P 9fdf5d1978a54d9fab8f80da1b992446 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a40ee039ec9e4a028b77e1ac82ff52b5" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 36095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:09:02.576843 23138 consensus_queue.cc:1035] T 81139f9dbf23419a809572772dd05720 P 9fdf5d1978a54d9fab8f80da1b992446 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a1e349d9f014f0fa2831b96bcb9ee66" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 37067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:09:02.579181 23145 mvcc.cc:205] Tried to move back new op lower bound from 7333372487989899264 to 7333372487950381056. Current Snapshot: MvccSnapshot[applied={T|T < 7333372487989899264}]
I20260925 22:09:02.579799 23146 mvcc.cc:205] Tried to move back new op lower bound from 7333372487989899264 to 7333372487950381056. Current Snapshot: MvccSnapshot[applied={T|T < 7333372487989899264 or (T in {7333372487989899264})}]
I20260925 22:09:02.580015 23147 mvcc.cc:205] Tried to move back new op lower bound from 7333372487989899264 to 7333372487950381056. Current Snapshot: MvccSnapshot[applied={T|T < 7333372487989899264}]
I20260925 22:09:02.581887 22668 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:09:02.581979 22668 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:09:02.582804 22668 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dd3989f71292422abd5dc3466c214a7a: Sending DeleteTablet for 3 replicas of tablet 81139f9dbf23419a809572772dd05720
I20260925 22:09:02.583181 22921 tablet_service.cc:1514] Processing DeleteTablet for tablet 81139f9dbf23419a809572772dd05720 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:09:02 UTC) from {username='slave'} at 127.0.0.1:35852
I20260925 22:09:02.583194 23052 tablet_service.cc:1514] Processing DeleteTablet for tablet 81139f9dbf23419a809572772dd05720 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:09:02 UTC) from {username='slave'} at 127.0.0.1:59536
I20260925 22:09:02.583202 22790 tablet_service.cc:1514] Processing DeleteTablet for tablet 81139f9dbf23419a809572772dd05720 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:09:02 UTC) from {username='slave'} at 127.0.0.1:33964
I20260925 22:09:02.583432 23154 tablet_replica.cc:330] T 81139f9dbf23419a809572772dd05720 P 3a1e349d9f014f0fa2831b96bcb9ee66: stopping tablet replica
I20260925 22:09:02.583532 23154 raft_consensus.cc:2229] T 81139f9dbf23419a809572772dd05720 P 3a1e349d9f014f0fa2831b96bcb9ee66 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:09:02.583566 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 22728
I20260925 22:09:02.583638 23154 raft_consensus.cc:2258] T 81139f9dbf23419a809572772dd05720 P 3a1e349d9f014f0fa2831b96bcb9ee66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:09:02.584149 23156 tablet_replica.cc:330] T 81139f9dbf23419a809572772dd05720 P a40ee039ec9e4a028b77e1ac82ff52b5: stopping tablet replica
I20260925 22:09:02.584233 23156 raft_consensus.cc:2229] T 81139f9dbf23419a809572772dd05720 P a40ee039ec9e4a028b77e1ac82ff52b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:09:02.584321 23156 raft_consensus.cc:2258] T 81139f9dbf23419a809572772dd05720 P a40ee039ec9e4a028b77e1ac82ff52b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:09:02.584748 23156 ts_tablet_manager.cc:1905] T 81139f9dbf23419a809572772dd05720 P a40ee039ec9e4a028b77e1ac82ff52b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:09:02.586534 23156 ts_tablet_manager.cc:1918] T 81139f9dbf23419a809572772dd05720 P a40ee039ec9e4a028b77e1ac82ff52b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:09:02.586625 23156 log.cc:1196] T 81139f9dbf23419a809572772dd05720 P a40ee039ec9e4a028b77e1ac82ff52b5: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-1/wal/wals/81139f9dbf23419a809572772dd05720
I20260925 22:09:02.586869 23156 ts_tablet_manager.cc:1939] T 81139f9dbf23419a809572772dd05720 P a40ee039ec9e4a028b77e1ac82ff52b5: Deleting consensus metadata
I20260925 22:09:02.587285 22652 catalog_manager.cc:4841] TS a40ee039ec9e4a028b77e1ac82ff52b5 (127.5.0.66:36095): tablet 81139f9dbf23419a809572772dd05720 (table test-table [id=a2b714982ef04406b8aefb85f125c906]) successfully deleted
I20260925 22:09:02.588021 23155 tablet_replica.cc:330] T 81139f9dbf23419a809572772dd05720 P 9fdf5d1978a54d9fab8f80da1b992446: stopping tablet replica
I20260925 22:09:02.588129 23155 raft_consensus.cc:2229] T 81139f9dbf23419a809572772dd05720 P 9fdf5d1978a54d9fab8f80da1b992446 [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:09:02.588251 23155 raft_consensus.cc:2258] T 81139f9dbf23419a809572772dd05720 P 9fdf5d1978a54d9fab8f80da1b992446 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:09:02.588682 23155 ts_tablet_manager.cc:1905] T 81139f9dbf23419a809572772dd05720 P 9fdf5d1978a54d9fab8f80da1b992446: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:09:02.590237 23155 ts_tablet_manager.cc:1918] T 81139f9dbf23419a809572772dd05720 P 9fdf5d1978a54d9fab8f80da1b992446: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:09:02.590318 23155 log.cc:1196] T 81139f9dbf23419a809572772dd05720 P 9fdf5d1978a54d9fab8f80da1b992446: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373908987421-5121-0/minicluster-data/ts-2/wal/wals/81139f9dbf23419a809572772dd05720
I20260925 22:09:02.590574 23155 ts_tablet_manager.cc:1939] T 81139f9dbf23419a809572772dd05720 P 9fdf5d1978a54d9fab8f80da1b992446: Deleting consensus metadata
I20260925 22:09:02.590953 22654 catalog_manager.cc:4841] TS 9fdf5d1978a54d9fab8f80da1b992446 (127.5.0.67:33029): tablet 81139f9dbf23419a809572772dd05720 (table test-table [id=a2b714982ef04406b8aefb85f125c906]) successfully deleted
W20260925 22:09:02.591228 22654 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.0.65:37067 (error 108)
I20260925 22:09:02.591569 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 22859
I20260925 22:09:02.591676 23126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:02.591548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:37067 (local address 127.0.0.1:33968)
0925 22:09:02.591595 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:02.591643 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.65:37067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260925 22:09:02.596022 22654 catalog_manager.cc:4584] TS 3a1e349d9f014f0fa2831b96bcb9ee66 (127.5.0.65:37067): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 81139f9dbf23419a809572772dd05720: Network error: Client connection negotiation failed: client connection to 127.5.0.65:37067: connect: Connection refused (error 111)
I20260925 22:09:02.597510 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 22990
I20260925 22:09:02.602231 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 22637
2026-09-25T22:09:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (685 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:09:02 dist-test-slave-7mqr krb5kdc[23160](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:09:02 dist-test-slave-7mqr krb5kdc[23160](info): set up 2 sockets
Sep 25 22:09:02 dist-test-slave-7mqr krb5kdc[23160](info): commencing operation
krb5kdc: starting...
W20260925 22:09:03.635730 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:09:03 dist-test-slave-7mqr krb5kdc[23160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:09:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:09:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.0.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
I20260925 22:09:03.755292 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:33529
--webserver_interface=127.5.0.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:43329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:33529
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:09:03.835203 23173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:03.835448 23173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:03.835482 23173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:03.836647 23173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:09:03.836715 23173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:03.836740 23173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:03.836764 23173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:09:03.836808 23173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:09:03.838279 23173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:43329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:33529
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:33529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:03.838549 23173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:03.838774 23173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 22:09:03.843545 23173 server_base.cc:1034] running on GCE node
W20260925 22:09:03.843551 23178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:03.843538 23181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:03.843659 23179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:03.844054 23173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:03.844338 23173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:03.845522 23173 hybrid_clock.cc:648] HybridClock initialized: now 1790374143845518 us; error 39 us; skew 500 ppm
Sep 25 22:09:03 dist-test-slave-7mqr krb5kdc[23160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374143, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:03.847959 23173 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:09:03.848404 23173 webserver.cc:460] Webserver started at http://127.5.0.126:35507/ using document root <none> and password file <none>
I20260925 22:09:03.848591 23173 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:03.848631 23173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:03.848717 23173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:03.849723 23173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "aac373204e5d4cd4a598e9bf2b2ff7a4"
format_stamp: "Formatted at 2026-09-25 22:09:03 on dist-test-slave-7mqr"
server_key: "3265fac8ce5ccfb129fe27ddc8a2b4dd"
server_key_iv: "0b8cf60a559ef8b5af0954eca0ee9d24"
server_key_version: "encryptionkey@0"
I20260925 22:09:03.850016 23173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "aac373204e5d4cd4a598e9bf2b2ff7a4"
format_stamp: "Formatted at 2026-09-25 22:09:03 on dist-test-slave-7mqr"
server_key: "3265fac8ce5ccfb129fe27ddc8a2b4dd"
server_key_iv: "0b8cf60a559ef8b5af0954eca0ee9d24"
server_key_version: "encryptionkey@0"
I20260925 22:09:03.851234 23173 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:09:03.851979 23188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:03.852156 23173 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:09:03.852243 23173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "aac373204e5d4cd4a598e9bf2b2ff7a4"
format_stamp: "Formatted at 2026-09-25 22:09:03 on dist-test-slave-7mqr"
server_key: "3265fac8ce5ccfb129fe27ddc8a2b4dd"
server_key_iv: "0b8cf60a559ef8b5af0954eca0ee9d24"
server_key_version: "encryptionkey@0"
I20260925 22:09:03.852321 23173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:03.870831 23173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:03.873206 23173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:03.873358 23173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:03.877313 23173 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:33529
I20260925 22:09:03.877375 23240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:33529 every 8 connection(s)
I20260925 22:09:03.877740 23173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:09:03.878638 23241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:03.880141 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 23173
I20260925 22:09:03.880232 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:09:03.880450 5121 external_mini_cluster.cc:1428] Setting key 184fd0e2e476e59b03d40df7e2889ef7
I20260925 22:09:03.881271 23241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aac373204e5d4cd4a598e9bf2b2ff7a4: Bootstrap starting.
I20260925 22:09:03.881937 23241 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aac373204e5d4cd4a598e9bf2b2ff7a4: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:03.882246 23241 log.cc:824] T 00000000000000000000000000000000 P aac373204e5d4cd4a598e9bf2b2ff7a4: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:03.882927 23241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aac373204e5d4cd4a598e9bf2b2ff7a4: No bootstrap required, opened a new log
I20260925 22:09:03.884749 23241 raft_consensus.cc:348] T 00000000000000000000000000000000 P aac373204e5d4cd4a598e9bf2b2ff7a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aac373204e5d4cd4a598e9bf2b2ff7a4" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 33529 } }
I20260925 22:09:03.884914 23241 raft_consensus.cc:374] T 00000000000000000000000000000000 P aac373204e5d4cd4a598e9bf2b2ff7a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:03.884959 23241 raft_consensus.cc:729] T 00000000000000000000000000000000 P aac373204e5d4cd4a598e9bf2b2ff7a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aac373204e5d4cd4a598e9bf2b2ff7a4, State: Initialized, Role: FOLLOWER
I20260925 22:09:03.885116 23241 consensus_queue.cc:260] T 00000000000000000000000000000000 P aac373204e5d4cd4a598e9bf2b2ff7a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aac373204e5d4cd4a598e9bf2b2ff7a4" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 33529 } }
I20260925 22:09:03.885212 23241 raft_consensus.cc:388] T 00000000000000000000000000000000 P aac373204e5d4cd4a598e9bf2b2ff7a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:09:03.885257 23241 raft_consensus.cc:482] T 00000000000000000000000000000000 P aac373204e5d4cd4a598e9bf2b2ff7a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:09:03.885301 23241 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aac373204e5d4cd4a598e9bf2b2ff7a4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:03.885921 23241 raft_consensus.cc:504] T 00000000000000000000000000000000 P aac373204e5d4cd4a598e9bf2b2ff7a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aac373204e5d4cd4a598e9bf2b2ff7a4" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 33529 } }
I20260925 22:09:03.886051 23241 leader_election.cc:302] T 00000000000000000000000000000000 P aac373204e5d4cd4a598e9bf2b2ff7a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aac373204e5d4cd4a598e9bf2b2ff7a4; no voters:
I20260925 22:09:03.886263 23241 leader_election.cc:288] T 00000000000000000000000000000000 P aac373204e5d4cd4a598e9bf2b2ff7a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:09:03.886289 23246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aac373204e5d4cd4a598e9bf2b2ff7a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:09:03.886422 23246 raft_consensus.cc:686] T 00000000000000000000000000000000 P aac373204e5d4cd4a598e9bf2b2ff7a4 [term 1 LEADER]: Becoming Leader. State: Replica: aac373204e5d4cd4a598e9bf2b2ff7a4, State: Running, Role: LEADER
I20260925 22:09:03.886515 23246 consensus_queue.cc:237] T 00000000000000000000000000000000 P aac373204e5d4cd4a598e9bf2b2ff7a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aac373204e5d4cd4a598e9bf2b2ff7a4" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 33529 } }
I20260925 22:09:03.886547 23241 sys_catalog.cc:564] T 00000000000000000000000000000000 P aac373204e5d4cd4a598e9bf2b2ff7a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:09:03.886830 23246 sys_catalog.cc:455] T 00000000000000000000000000000000 P aac373204e5d4cd4a598e9bf2b2ff7a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aac373204e5d4cd4a598e9bf2b2ff7a4. Latest consensus state: current_term: 1 leader_uuid: "aac373204e5d4cd4a598e9bf2b2ff7a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aac373204e5d4cd4a598e9bf2b2ff7a4" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 33529 } } }
I20260925 22:09:03.886915 23246 sys_catalog.cc:458] T 00000000000000000000000000000000 P aac373204e5d4cd4a598e9bf2b2ff7a4 [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:03.886844 23248 sys_catalog.cc:455] T 00000000000000000000000000000000 P aac373204e5d4cd4a598e9bf2b2ff7a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aac373204e5d4cd4a598e9bf2b2ff7a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aac373204e5d4cd4a598e9bf2b2ff7a4" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 33529 } } }
I20260925 22:09:03.887348 23248 sys_catalog.cc:458] T 00000000000000000000000000000000 P aac373204e5d4cd4a598e9bf2b2ff7a4 [sys.catalog]: This master's current role is: LEADER
Sep 25 22:09:03 dist-test-slave-7mqr krb5kdc[23160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:09:03.887693 23255 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:09:03.888330 23255 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:09:03.890692 23255 catalog_manager.cc:1347] Generated new cluster ID: 845ae005f6e44a528600b3dfb64ba8f8
I20260925 22:09:03.890755 23255 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:09:03.891995 23244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:03.880984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51860 (local address 127.5.0.126:33529)
0925 22:09:03.881177 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:03.881193 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:03.881218 (+ 25us) server_negotiation.cc:407] Connection header received
0925 22:09:03.881705 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:03.881714 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:03.881773 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:03.881919 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:03.882622 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:03.883515 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:03.884339 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:03.884761 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:03.884900 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:03.888163 (+ 3263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:03.888168 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:03.888168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:03.888181 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:03.890312 (+ 2131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:03.890688 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:03.890691 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:03.890691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:03.890852 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:03.891063 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:03.891064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:03.891065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:03.891317 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:03.891476 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:03.891601 (+ 125us) server_negotiation.cc:299] Negotiation successful
0925 22:09:03.891750 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":91,"threads_started":1}
I20260925 22:09:03.921586 23255 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:09:03.922353 23255 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:09:03.926672 23255 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aac373204e5d4cd4a598e9bf2b2ff7a4: Generated new TSK 0
I20260925 22:09:03.926874 23255 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.0.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
I20260925 22:09:03.960903 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.65:0
--local_ip_for_outbound_sockets=127.5.0.65
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:33529
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:43329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:09:04.045209 23267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:04.045401 23267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:04.045425 23267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:04.046593 23267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:04.046638 23267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:04.046662 23267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.65
I20260925 22:09:04.048177 23267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:43329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:33529
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.65
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:04.048434 23267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:04.048662 23267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:04.053431 23275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:04.053432 23272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:04.053447 23273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:04.053777 23267 server_base.cc:1034] running on GCE node
I20260925 22:09:04.053983 23267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:04.054252 23267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:04.055398 23267 hybrid_clock.cc:648] HybridClock initialized: now 1790374144055372 us; error 37 us; skew 500 ppm
Sep 25 22:09:04 dist-test-slave-7mqr krb5kdc[23160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374144, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:04.057988 23267 init.cc:377] Logged in from keytab as kudu/127.5.0.65@KRBTEST.COM (short username kudu)
I20260925 22:09:04.058405 23267 webserver.cc:460] Webserver started at http://127.5.0.65:39071/ using document root <none> and password file <none>
I20260925 22:09:04.058650 23267 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:04.058722 23267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:04.058868 23267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:04.059962 23267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-0/data/instance:
uuid: "9903e7e451bd43eeb861f8d13cfcecfd"
format_stamp: "Formatted at 2026-09-25 22:09:04 on dist-test-slave-7mqr"
server_key: "c3de979d3f5a50f271a96078727893cf"
server_key_iv: "11a1ee397b55a6b404d948bec76ddf93"
server_key_version: "encryptionkey@0"
I20260925 22:09:04.060323 23267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance:
uuid: "9903e7e451bd43eeb861f8d13cfcecfd"
format_stamp: "Formatted at 2026-09-25 22:09:04 on dist-test-slave-7mqr"
server_key: "c3de979d3f5a50f271a96078727893cf"
server_key_iv: "11a1ee397b55a6b404d948bec76ddf93"
server_key_version: "encryptionkey@0"
I20260925 22:09:04.061648 23267 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:09:04.062412 23282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:04.062602 23267 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:09:04.062700 23267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-0/wal
uuid: "9903e7e451bd43eeb861f8d13cfcecfd"
format_stamp: "Formatted at 2026-09-25 22:09:04 on dist-test-slave-7mqr"
server_key: "c3de979d3f5a50f271a96078727893cf"
server_key_iv: "11a1ee397b55a6b404d948bec76ddf93"
server_key_version: "encryptionkey@0"
I20260925 22:09:04.062783 23267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:04.073251 23267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:04.075554 23267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:04.075721 23267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:04.075989 23267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:09:04.076303 23267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:09:04.076357 23267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:04.076398 23267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:09:04.076426 23267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:04.081933 23267 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.65:40041
I20260925 22:09:04.082022 23395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.65:40041 every 8 connection(s)
I20260925 22:09:04.082331 23267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
I20260925 22:09:04.085636 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 23267
I20260925 22:09:04.085758 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance
I20260925 22:09:04.085971 5121 external_mini_cluster.cc:1428] Setting key e9f4bdb715707ad85b834a525852b9e5
Sep 25 22:09:04 dist-test-slave-7mqr krb5kdc[23160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374144, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:09:04.092550 23244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:04.083065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:36739 (local address 127.5.0.126:33529)
0925 22:09:04.083146 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:04.083149 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:04.083348 (+ 199us) server_negotiation.cc:407] Connection header received
0925 22:09:04.083809 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:04.083811 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:04.083820 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:04.083867 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:04.084844 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.085343 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.086722 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.087026 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.087121 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:04.089564 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:04.089566 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:04.089566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:04.089574 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:04.091050 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:04.091492 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:04.091493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:04.091493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:04.091552 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:04.091816 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:04.091817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:04.091817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:04.091996 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:04.092071 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:04.092344 (+ 273us) server_negotiation.cc:299] Negotiation successful
0925 22:09:04.092441 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 22:09:04.092712 23398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:04.083045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:33529 (local address 127.5.0.65:36739)
0925 22:09:04.083230 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:04.083266 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:04.083731 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:04.083977 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:04.083981 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:04.084577 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:04.084766 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:04.084771 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.085451 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.085452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:04.086630 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:04.086633 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.087158 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.087159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:04.087314 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:04.087883 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:09:04.087929 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:04.089466 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:04.091154 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:04.091161 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:04.091163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:04.091394 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:04.091629 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:04.091630 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:04.091630 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:04.091743 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:04.092062 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:04.092069 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:04.092212 (+ 143us) client_negotiation.cc:769] Sending connection context
0925 22:09:04.092323 (+ 111us) client_negotiation.cc:240] Negotiation successful
0925 22:09:04.092480 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":104,"threads_started":1}
I20260925 22:09:04.093307 23396 heartbeater.cc:347] Connected to a master server at 127.5.0.126:33529
I20260925 22:09:04.093408 23396 heartbeater.cc:464] Registering TS with master...
I20260925 22:09:04.093614 23396 heartbeater.cc:511] Master 127.5.0.126:33529 requested a full tablet report, sending...
I20260925 22:09:04.094277 23205 ts_manager.cc:194] Registered new tserver with Master: 9903e7e451bd43eeb861f8d13cfcecfd (127.5.0.65:40041)
WARNING: no policy specified for kudu/127.5.0.66@KRBTEST.COM; defaulting to no policy
I20260925 22:09:04.095237 23205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.65@KRBTEST.COM'} at 127.5.0.65:36739
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
I20260925 22:09:04.113122 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.66:0
--local_ip_for_outbound_sockets=127.5.0.66
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:33529
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:43329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:09:04.191934 23401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:04.192140 23401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:04.192176 23401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:04.193272 23401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:04.193336 23401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:04.193382 23401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.66
I20260925 22:09:04.194793 23401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:43329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:33529
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.66
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:04.195051 23401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:04.195271 23401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:04.200006 23407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:04.200024 23406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:04.200013 23409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:04.200263 23401 server_base.cc:1034] running on GCE node
I20260925 22:09:04.200413 23401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:04.200598 23401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:04.201737 23401 hybrid_clock.cc:648] HybridClock initialized: now 1790374144201726 us; error 25 us; skew 500 ppm
Sep 25 22:09:04 dist-test-slave-7mqr krb5kdc[23160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374144, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:04.204147 23401 init.cc:377] Logged in from keytab as kudu/127.5.0.66@KRBTEST.COM (short username kudu)
I20260925 22:09:04.204553 23401 webserver.cc:460] Webserver started at http://127.5.0.66:36507/ using document root <none> and password file <none>
I20260925 22:09:04.204784 23401 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:04.204828 23401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:04.204968 23401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:04.205968 23401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-1/data/instance:
uuid: "ba8d12ca459449179b42f0886b2993bd"
format_stamp: "Formatted at 2026-09-25 22:09:04 on dist-test-slave-7mqr"
server_key: "496659b92136135adfdbefda6166d4a3"
server_key_iv: "83fd45991153ff5d173885e3af8bcd34"
server_key_version: "encryptionkey@0"
I20260925 22:09:04.206317 23401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance:
uuid: "ba8d12ca459449179b42f0886b2993bd"
format_stamp: "Formatted at 2026-09-25 22:09:04 on dist-test-slave-7mqr"
server_key: "496659b92136135adfdbefda6166d4a3"
server_key_iv: "83fd45991153ff5d173885e3af8bcd34"
server_key_version: "encryptionkey@0"
I20260925 22:09:04.207671 23401 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:09:04.208436 23416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:04.208607 23401 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:04.208707 23401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-1/wal
uuid: "ba8d12ca459449179b42f0886b2993bd"
format_stamp: "Formatted at 2026-09-25 22:09:04 on dist-test-slave-7mqr"
server_key: "496659b92136135adfdbefda6166d4a3"
server_key_iv: "83fd45991153ff5d173885e3af8bcd34"
server_key_version: "encryptionkey@0"
I20260925 22:09:04.208786 23401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:04.219099 23401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:04.220930 23401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:04.221091 23401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:04.221297 23401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:09:04.221578 23401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:09:04.221632 23401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:04.221673 23401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:09:04.221700 23401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:04.227174 23401 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.66:35773
I20260925 22:09:04.227242 23529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.66:35773 every 8 connection(s)
I20260925 22:09:04.227577 23401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
Sep 25 22:09:04 dist-test-slave-7mqr krb5kdc[23160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374144, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:09:04.236919 23244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:04.228337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:54771 (local address 127.5.0.126:33529)
0925 22:09:04.228404 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:04.228406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:04.228562 (+ 156us) server_negotiation.cc:407] Connection header received
0925 22:09:04.229081 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:04.229082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:04.229096 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:04.229142 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:04.230090 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.230613 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.231622 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.231825 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.231921 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:04.234061 (+ 2140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:04.234062 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:04.234063 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:04.234070 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:04.235517 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:04.236005 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:04.236006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:04.236006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:04.236067 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:04.236331 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:04.236332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:04.236332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:04.236489 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:04.236544 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:04.236781 (+ 237us) server_negotiation.cc:299] Negotiation successful
0925 22:09:04.236850 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 22:09:04.237138 23532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:04.228284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:33529 (local address 127.5.0.66:54771)
0925 22:09:04.228440 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:04.228479 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:04.228956 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:04.229228 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:04.229231 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:04.229775 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:04.229964 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:04.229969 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.230710 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.230711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:04.231523 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:04.231526 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.231935 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.231937 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:04.232064 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:04.232546 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:09:04.232554 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:04.233969 (+ 1415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:04.235597 (+ 1628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:04.235602 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:04.235604 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:04.235829 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:04.236158 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:04.236159 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:04.236159 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:04.236265 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:04.236551 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:04.236555 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:04.236670 (+ 115us) client_negotiation.cc:769] Sending connection context
0925 22:09:04.236754 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 22:09:04.236877 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":86,"threads_started":1}
I20260925 22:09:04.237450 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 23401
I20260925 22:09:04.237589 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance
I20260925 22:09:04.237766 23530 heartbeater.cc:347] Connected to a master server at 127.5.0.126:33529
I20260925 22:09:04.237836 5121 external_mini_cluster.cc:1428] Setting key 634c73930b1c3970f5f1c5f04b4cfe89
I20260925 22:09:04.237874 23530 heartbeater.cc:464] Registering TS with master...
I20260925 22:09:04.238070 23530 heartbeater.cc:511] Master 127.5.0.126:33529 requested a full tablet report, sending...
I20260925 22:09:04.238440 23205 ts_manager.cc:194] Registered new tserver with Master: ba8d12ca459449179b42f0886b2993bd (127.5.0.66:35773)
I20260925 22:09:04.238941 23205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.66@KRBTEST.COM'} at 127.5.0.66:54771
WARNING: no policy specified for kudu/127.5.0.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
I20260925 22:09:04.262141 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.67:0
--local_ip_for_outbound_sockets=127.5.0.67
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:33529
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:43329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:09:04.341392 23535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:04.341596 23535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:04.341619 23535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:04.342751 23535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:04.342792 23535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:04.342845 23535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.67
I20260925 22:09:04.344264 23535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:43329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:33529
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.67
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:04.344502 23535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:04.344734 23535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:04.349614 23543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:04.349614 23540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:04.349773 23535 server_base.cc:1034] running on GCE node
W20260925 22:09:04.349713 23541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:04.350076 23535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:04.350274 23535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:04.351449 23535 hybrid_clock.cc:648] HybridClock initialized: now 1790374144351405 us; error 33 us; skew 500 ppm
Sep 25 22:09:04 dist-test-slave-7mqr krb5kdc[23160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374144, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:04.353896 23535 init.cc:377] Logged in from keytab as kudu/127.5.0.67@KRBTEST.COM (short username kudu)
I20260925 22:09:04.354388 23535 webserver.cc:460] Webserver started at http://127.5.0.67:41503/ using document root <none> and password file <none>
I20260925 22:09:04.354622 23535 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:04.354691 23535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:04.354817 23535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:04.355960 23535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-2/data/instance:
uuid: "c2caed2e06a7409c94332d0539fc9b97"
format_stamp: "Formatted at 2026-09-25 22:09:04 on dist-test-slave-7mqr"
server_key: "682dc302355a61d9e49b9f85102b7908"
server_key_iv: "f25ded635903262a465f61a35a2d56a4"
server_key_version: "encryptionkey@0"
I20260925 22:09:04.356302 23535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance:
uuid: "c2caed2e06a7409c94332d0539fc9b97"
format_stamp: "Formatted at 2026-09-25 22:09:04 on dist-test-slave-7mqr"
server_key: "682dc302355a61d9e49b9f85102b7908"
server_key_iv: "f25ded635903262a465f61a35a2d56a4"
server_key_version: "encryptionkey@0"
I20260925 22:09:04.357537 23535 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:09:04.358198 23550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:04.358366 23535 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:09:04.358444 23535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-2/wal
uuid: "c2caed2e06a7409c94332d0539fc9b97"
format_stamp: "Formatted at 2026-09-25 22:09:04 on dist-test-slave-7mqr"
server_key: "682dc302355a61d9e49b9f85102b7908"
server_key_iv: "f25ded635903262a465f61a35a2d56a4"
server_key_version: "encryptionkey@0"
I20260925 22:09:04.358501 23535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:04.374804 23535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:04.376729 23535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:04.376837 23535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:04.377017 23535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:09:04.377316 23535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:09:04.377348 23535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:04.377372 23535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:09:04.377419 23535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:04.382838 23535 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.67:40261
I20260925 22:09:04.382962 23663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.67:40261 every 8 connection(s)
I20260925 22:09:04.383244 23535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
I20260925 22:09:04.386317 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 23535
I20260925 22:09:04.386466 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance
I20260925 22:09:04.386740 5121 external_mini_cluster.cc:1428] Setting key 4207e9281f704bf3ceb1b5af3a015322
Sep 25 22:09:04 dist-test-slave-7mqr krb5kdc[23160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374144, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:09:04.392571 23244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:04.383937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:48673 (local address 127.5.0.126:33529)
0925 22:09:04.383981 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:04.383984 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:04.384211 (+ 227us) server_negotiation.cc:407] Connection header received
0925 22:09:04.384719 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:04.384720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:04.384732 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:04.384797 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:04.385706 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.386318 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.387912 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.388139 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.388197 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:04.390204 (+ 2007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:04.390205 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:04.390205 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:04.390210 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:04.391346 (+ 1136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:04.391700 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:04.391700 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:04.391701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:04.391746 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:04.392017 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:04.392018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:04.392018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:04.392152 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:04.392212 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:04.392441 (+ 229us) server_negotiation.cc:299] Negotiation successful
0925 22:09:04.392507 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:09:04.392822 23666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:04.383938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:33529 (local address 127.5.0.67:48673)
0925 22:09:04.384088 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:04.384123 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:04.384611 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:04.384870 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:04.384873 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:04.385418 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:04.385603 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:04.385607 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.386459 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.386460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:04.387817 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:04.387820 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.388214 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.388215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:04.388311 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:04.388808 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:09:04.388824 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:04.390127 (+ 1303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:04.391415 (+ 1288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:04.391421 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:04.391423 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:04.391637 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:04.391803 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:04.391804 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:04.391804 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:04.391941 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:04.392217 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:04.392221 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:04.392330 (+ 109us) client_negotiation.cc:769] Sending connection context
0925 22:09:04.392412 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 22:09:04.392545 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":72,"threads_started":1}
I20260925 22:09:04.393639 23664 heartbeater.cc:347] Connected to a master server at 127.5.0.126:33529
I20260925 22:09:04.393719 23664 heartbeater.cc:464] Registering TS with master...
I20260925 22:09:04.393901 23664 heartbeater.cc:511] Master 127.5.0.126:33529 requested a full tablet report, sending...
I20260925 22:09:04.394270 23205 ts_manager.cc:194] Registered new tserver with Master: c2caed2e06a7409c94332d0539fc9b97 (127.5.0.67:40261)
I20260925 22:09:04.394639 23205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.67@KRBTEST.COM'} at 127.5.0.67:48673
I20260925 22:09:04.398458 5121 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:09:04.405223 23244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:04.399770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51870 (local address 127.5.0.126:33529)
0925 22:09:04.399818 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:04.399820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:04.399954 (+ 134us) server_negotiation.cc:407] Connection header received
0925 22:09:04.400013 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:04.400014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:04.400022 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:04.400073 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:04.400563 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.401015 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.401668 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.401833 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.401879 (+ 46us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:04.402977 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:04.402978 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:04.402979 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:04.402984 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:04.404224 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:04.404569 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:04.404570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:04.404570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:04.404617 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:04.404760 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:04.404761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:04.404761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:04.404905 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:04.404988 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:04.405117 (+ 129us) server_negotiation.cc:299] Negotiation successful
0925 22:09:04.405165 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260925 22:09:04.406667 23205 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51870:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:09:04.408259 23205 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:09:04.414022 23673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:04.410134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:40041 (local address 127.0.0.1:44650)
0925 22:09:04.410282 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:04.410319 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:04.410379 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:04.410660 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:04.410663 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:04.410665 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:09:04.410737 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:04.410739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.411860 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.411861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:04.412827 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:04.412830 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.413686 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.413687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:04.413772 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:04.413785 (+ 13us) client_negotiation.cc:769] Sending connection context
0925 22:09:04.413875 (+ 90us) client_negotiation.cc:240] Negotiation successful
0925 22:09:04.413934 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":51,"threads_started":1}
I20260925 22:09:04.414079 23675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:04.410293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51830 (local address 127.5.0.67:40261)
0925 22:09:04.410784 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:04.410788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:04.410799 (+ 11us) server_negotiation.cc:407] Connection header received
0925 22:09:04.410831 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:04.410833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:04.410845 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:04.410942 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:09:04.411091 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.412088 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.413054 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.413765 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.413839 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:04.413885 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:04.413970 (+ 85us) server_negotiation.cc:299] Negotiation successful
0925 22:09:04.414003 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":83,"threads_started":1}
I20260925 22:09:04.414095 23672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:04.410172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:40261 (local address 127.0.0.1:51830)
0925 22:09:04.410462 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:04.410474 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:04.410529 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:04.410939 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:04.410940 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:04.410942 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:09:04.410984 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:04.410986 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.412174 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.412175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:04.412967 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:04.412969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.413848 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.413849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:04.413909 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:04.413912 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:09:04.413962 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 22:09:04.413993 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":65,"threads_started":1}
I20260925 22:09:04.414371 23676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:04.410292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44650 (local address 127.5.0.65:40041)
0925 22:09:04.410477 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:04.410482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:04.410495 (+ 13us) server_negotiation.cc:407] Connection header received
0925 22:09:04.410564 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:04.410566 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:04.410582 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:04.410658 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:09:04.410805 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.411757 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.412930 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.413600 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.413688 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:04.413748 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:04.414227 (+ 479us) server_negotiation.cc:299] Negotiation successful
0925 22:09:04.414273 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":86,"threads_started":1}
I20260925 22:09:04.414947 23598 tablet_service.cc:1467] Processing CreateTablet for tablet 68c14fe5479b41b4a73f63844308d4e5 (DEFAULT_TABLE table=test-table [id=f1a1498e1bae45d59a61c242188f5548]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:09:04.414985 23330 tablet_service.cc:1467] Processing CreateTablet for tablet 68c14fe5479b41b4a73f63844308d4e5 (DEFAULT_TABLE table=test-table [id=f1a1498e1bae45d59a61c242188f5548]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:09:04.415287 23598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68c14fe5479b41b4a73f63844308d4e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:04.415309 23330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68c14fe5479b41b4a73f63844308d4e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:04.416893 23674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:04.410312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:35773 (local address 127.0.0.1:37902)
0925 22:09:04.410686 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:04.410696 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:04.410749 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:04.412038 (+ 1289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:04.412039 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:04.412041 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:09:04.412111 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:04.412112 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.414489 (+ 2377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.414490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:04.415660 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:04.415662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.416661 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.416662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:04.416726 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:04.416729 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:09:04.416778 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 22:09:04.416817 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":22,"thread_start_us":38,"threads_started":1}
I20260925 22:09:04.417276 23677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:04.410504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37902 (local address 127.5.0.66:35773)
0925 22:09:04.410670 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:04.410674 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:04.410738 (+ 64us) server_negotiation.cc:407] Connection header received
0925 22:09:04.411925 (+ 1187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:04.411929 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:04.411944 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:04.412026 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:09:04.413085 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.414252 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.415759 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.416498 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.416569 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:04.416624 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:04.417140 (+ 516us) server_negotiation.cc:299] Negotiation successful
0925 22:09:04.417192 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":97,"threads_started":1}
I20260925 22:09:04.417908 23464 tablet_service.cc:1467] Processing CreateTablet for tablet 68c14fe5479b41b4a73f63844308d4e5 (DEFAULT_TABLE table=test-table [id=f1a1498e1bae45d59a61c242188f5548]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:09:04.418064 23678 tablet_bootstrap.cc:492] T 68c14fe5479b41b4a73f63844308d4e5 P c2caed2e06a7409c94332d0539fc9b97: Bootstrap starting.
I20260925 22:09:04.418217 23464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68c14fe5479b41b4a73f63844308d4e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:04.418529 23679 tablet_bootstrap.cc:492] T 68c14fe5479b41b4a73f63844308d4e5 P 9903e7e451bd43eeb861f8d13cfcecfd: Bootstrap starting.
I20260925 22:09:04.418809 23678 tablet_bootstrap.cc:654] T 68c14fe5479b41b4a73f63844308d4e5 P c2caed2e06a7409c94332d0539fc9b97: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:04.419194 23679 tablet_bootstrap.cc:654] T 68c14fe5479b41b4a73f63844308d4e5 P 9903e7e451bd43eeb861f8d13cfcecfd: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:04.419404 23678 log.cc:824] T 68c14fe5479b41b4a73f63844308d4e5 P c2caed2e06a7409c94332d0539fc9b97: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:04.419732 23679 log.cc:824] T 68c14fe5479b41b4a73f63844308d4e5 P 9903e7e451bd43eeb861f8d13cfcecfd: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:04.420234 23678 tablet_bootstrap.cc:492] T 68c14fe5479b41b4a73f63844308d4e5 P c2caed2e06a7409c94332d0539fc9b97: No bootstrap required, opened a new log
I20260925 22:09:04.420310 23678 ts_tablet_manager.cc:1397] T 68c14fe5479b41b4a73f63844308d4e5 P c2caed2e06a7409c94332d0539fc9b97: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:09:04.420418 23679 tablet_bootstrap.cc:492] T 68c14fe5479b41b4a73f63844308d4e5 P 9903e7e451bd43eeb861f8d13cfcecfd: No bootstrap required, opened a new log
I20260925 22:09:04.420495 23679 ts_tablet_manager.cc:1397] T 68c14fe5479b41b4a73f63844308d4e5 P 9903e7e451bd43eeb861f8d13cfcecfd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:09:04.421278 23682 tablet_bootstrap.cc:492] T 68c14fe5479b41b4a73f63844308d4e5 P ba8d12ca459449179b42f0886b2993bd: Bootstrap starting.
I20260925 22:09:04.421968 23682 tablet_bootstrap.cc:654] T 68c14fe5479b41b4a73f63844308d4e5 P ba8d12ca459449179b42f0886b2993bd: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:04.422115 23678 raft_consensus.cc:348] T 68c14fe5479b41b4a73f63844308d4e5 P c2caed2e06a7409c94332d0539fc9b97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9903e7e451bd43eeb861f8d13cfcecfd" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 40041 } } peers { permanent_uuid: "ba8d12ca459449179b42f0886b2993bd" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 35773 } } peers { permanent_uuid: "c2caed2e06a7409c94332d0539fc9b97" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 40261 } }
I20260925 22:09:04.422317 23679 raft_consensus.cc:348] T 68c14fe5479b41b4a73f63844308d4e5 P 9903e7e451bd43eeb861f8d13cfcecfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9903e7e451bd43eeb861f8d13cfcecfd" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 40041 } } peers { permanent_uuid: "ba8d12ca459449179b42f0886b2993bd" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 35773 } } peers { permanent_uuid: "c2caed2e06a7409c94332d0539fc9b97" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 40261 } }
I20260925 22:09:04.422492 23679 raft_consensus.cc:374] T 68c14fe5479b41b4a73f63844308d4e5 P 9903e7e451bd43eeb861f8d13cfcecfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:04.422479 23678 raft_consensus.cc:374] T 68c14fe5479b41b4a73f63844308d4e5 P c2caed2e06a7409c94332d0539fc9b97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:04.422538 23678 raft_consensus.cc:729] T 68c14fe5479b41b4a73f63844308d4e5 P c2caed2e06a7409c94332d0539fc9b97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2caed2e06a7409c94332d0539fc9b97, State: Initialized, Role: FOLLOWER
I20260925 22:09:04.422538 23679 raft_consensus.cc:729] T 68c14fe5479b41b4a73f63844308d4e5 P 9903e7e451bd43eeb861f8d13cfcecfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9903e7e451bd43eeb861f8d13cfcecfd, State: Initialized, Role: FOLLOWER
I20260925 22:09:04.422545 23682 log.cc:824] T 68c14fe5479b41b4a73f63844308d4e5 P ba8d12ca459449179b42f0886b2993bd: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:04.422672 23678 consensus_queue.cc:260] T 68c14fe5479b41b4a73f63844308d4e5 P c2caed2e06a7409c94332d0539fc9b97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9903e7e451bd43eeb861f8d13cfcecfd" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 40041 } } peers { permanent_uuid: "ba8d12ca459449179b42f0886b2993bd" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 35773 } } peers { permanent_uuid: "c2caed2e06a7409c94332d0539fc9b97" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 40261 } }
I20260925 22:09:04.422674 23679 consensus_queue.cc:260] T 68c14fe5479b41b4a73f63844308d4e5 P 9903e7e451bd43eeb861f8d13cfcecfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9903e7e451bd43eeb861f8d13cfcecfd" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 40041 } } peers { permanent_uuid: "ba8d12ca459449179b42f0886b2993bd" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 35773 } } peers { permanent_uuid: "c2caed2e06a7409c94332d0539fc9b97" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 40261 } }
I20260925 22:09:04.422981 23679 ts_tablet_manager.cc:1428] T 68c14fe5479b41b4a73f63844308d4e5 P 9903e7e451bd43eeb861f8d13cfcecfd: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 22:09:04.423002 23396 heartbeater.cc:503] Master 127.5.0.126:33529 was elected leader, sending a full tablet report...
I20260925 22:09:04.422948 23678 ts_tablet_manager.cc:1428] T 68c14fe5479b41b4a73f63844308d4e5 P c2caed2e06a7409c94332d0539fc9b97: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 22:09:04.423137 23664 heartbeater.cc:503] Master 127.5.0.126:33529 was elected leader, sending a full tablet report...
I20260925 22:09:04.423256 23682 tablet_bootstrap.cc:492] T 68c14fe5479b41b4a73f63844308d4e5 P ba8d12ca459449179b42f0886b2993bd: No bootstrap required, opened a new log
I20260925 22:09:04.423326 23682 ts_tablet_manager.cc:1397] T 68c14fe5479b41b4a73f63844308d4e5 P ba8d12ca459449179b42f0886b2993bd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:09:04.424922 23682 raft_consensus.cc:348] T 68c14fe5479b41b4a73f63844308d4e5 P ba8d12ca459449179b42f0886b2993bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9903e7e451bd43eeb861f8d13cfcecfd" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 40041 } } peers { permanent_uuid: "ba8d12ca459449179b42f0886b2993bd" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 35773 } } peers { permanent_uuid: "c2caed2e06a7409c94332d0539fc9b97" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 40261 } }
I20260925 22:09:04.425051 23682 raft_consensus.cc:374] T 68c14fe5479b41b4a73f63844308d4e5 P ba8d12ca459449179b42f0886b2993bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:04.425073 23682 raft_consensus.cc:729] T 68c14fe5479b41b4a73f63844308d4e5 P ba8d12ca459449179b42f0886b2993bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba8d12ca459449179b42f0886b2993bd, State: Initialized, Role: FOLLOWER
I20260925 22:09:04.425184 23682 consensus_queue.cc:260] T 68c14fe5479b41b4a73f63844308d4e5 P ba8d12ca459449179b42f0886b2993bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9903e7e451bd43eeb861f8d13cfcecfd" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 40041 } } peers { permanent_uuid: "ba8d12ca459449179b42f0886b2993bd" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 35773 } } peers { permanent_uuid: "c2caed2e06a7409c94332d0539fc9b97" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 40261 } }
I20260925 22:09:04.425449 23682 ts_tablet_manager.cc:1428] T 68c14fe5479b41b4a73f63844308d4e5 P ba8d12ca459449179b42f0886b2993bd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:09:04.425474 23530 heartbeater.cc:503] Master 127.5.0.126:33529 was elected leader, sending a full tablet report...
I20260925 22:09:04.459041 23685 raft_consensus.cc:482] T 68c14fe5479b41b4a73f63844308d4e5 P 9903e7e451bd43eeb861f8d13cfcecfd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:09:04.459200 23685 raft_consensus.cc:504] T 68c14fe5479b41b4a73f63844308d4e5 P 9903e7e451bd43eeb861f8d13cfcecfd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9903e7e451bd43eeb861f8d13cfcecfd" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 40041 } } peers { permanent_uuid: "ba8d12ca459449179b42f0886b2993bd" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 35773 } } peers { permanent_uuid: "c2caed2e06a7409c94332d0539fc9b97" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 40261 } }
I20260925 22:09:04.459613 23685 leader_election.cc:288] T 68c14fe5479b41b4a73f63844308d4e5 P 9903e7e451bd43eeb861f8d13cfcecfd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba8d12ca459449179b42f0886b2993bd (127.5.0.66:35773), c2caed2e06a7409c94332d0539fc9b97 (127.5.0.67:40261)
I20260925 22:09:04.463593 23675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:04.459872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:51593 (local address 127.5.0.67:40261)
0925 22:09:04.459986 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:04.459988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:04.460120 (+ 132us) server_negotiation.cc:407] Connection header received
0925 22:09:04.460166 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:04.460168 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:04.460177 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:04.460237 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:09:04.460357 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.461088 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.462597 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.463180 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.463256 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:04.463265 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:04.463479 (+ 214us) server_negotiation.cc:299] Negotiation successful
0925 22:09:04.463519 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260925 22:09:04.463593 23677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:04.459871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:55681 (local address 127.5.0.66:35773)
0925 22:09:04.459946 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:04.459948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:04.459958 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:09:04.459989 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:04.459990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:04.460003 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:04.460065 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:09:04.460250 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.461070 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.462596 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.463175 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.463256 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:04.463265 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:04.463479 (+ 214us) server_negotiation.cc:299] Negotiation successful
0925 22:09:04.463525 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 22:09:04.463596 23398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:04.459755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:35773 (local address 127.5.0.65:55681)
0925 22:09:04.459810 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:04.459822 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:04.459868 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:04.460100 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:04.460100 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:04.460103 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:09:04.460177 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:04.460178 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.461159 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.461160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:04.462501 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:04.462505 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.463293 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.463294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:04.463387 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:04.463393 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 22:09:04.463461 (+ 68us) client_negotiation.cc:240] Negotiation successful
0925 22:09:04.463519 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":35}
I20260925 22:09:04.463604 23687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:04.459759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:40261 (local address 127.5.0.65:51593)
0925 22:09:04.460031 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:04.460039 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:04.460107 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:04.460243 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:04.460244 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:04.460245 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:09:04.460299 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:04.460300 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.461159 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.461160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:04.462502 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:04.462505 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.463293 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.463294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:04.463387 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:04.463394 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 22:09:04.463461 (+ 67us) client_negotiation.cc:240] Negotiation successful
0925 22:09:04.463506 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":141,"thread_start_us":150,"threads_started":1}
I20260925 22:09:04.463950 23484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68c14fe5479b41b4a73f63844308d4e5" candidate_uuid: "9903e7e451bd43eeb861f8d13cfcecfd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba8d12ca459449179b42f0886b2993bd" is_pre_election: true
I20260925 22:09:04.464108 23484 raft_consensus.cc:2454] T 68c14fe5479b41b4a73f63844308d4e5 P ba8d12ca459449179b42f0886b2993bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9903e7e451bd43eeb861f8d13cfcecfd in term 0.
I20260925 22:09:04.464083 23618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68c14fe5479b41b4a73f63844308d4e5" candidate_uuid: "9903e7e451bd43eeb861f8d13cfcecfd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2caed2e06a7409c94332d0539fc9b97" is_pre_election: true
I20260925 22:09:04.464236 23618 raft_consensus.cc:2454] T 68c14fe5479b41b4a73f63844308d4e5 P c2caed2e06a7409c94332d0539fc9b97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9903e7e451bd43eeb861f8d13cfcecfd in term 0.
I20260925 22:09:04.464345 23286 leader_election.cc:302] T 68c14fe5479b41b4a73f63844308d4e5 P 9903e7e451bd43eeb861f8d13cfcecfd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9903e7e451bd43eeb861f8d13cfcecfd, ba8d12ca459449179b42f0886b2993bd; no voters:
I20260925 22:09:04.464485 23685 raft_consensus.cc:2781] T 68c14fe5479b41b4a73f63844308d4e5 P 9903e7e451bd43eeb861f8d13cfcecfd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:09:04.464553 23685 raft_consensus.cc:482] T 68c14fe5479b41b4a73f63844308d4e5 P 9903e7e451bd43eeb861f8d13cfcecfd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:09:04.464572 23685 raft_consensus.cc:3037] T 68c14fe5479b41b4a73f63844308d4e5 P 9903e7e451bd43eeb861f8d13cfcecfd [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:04.465345 23685 raft_consensus.cc:504] T 68c14fe5479b41b4a73f63844308d4e5 P 9903e7e451bd43eeb861f8d13cfcecfd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9903e7e451bd43eeb861f8d13cfcecfd" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 40041 } } peers { permanent_uuid: "ba8d12ca459449179b42f0886b2993bd" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 35773 } } peers { permanent_uuid: "c2caed2e06a7409c94332d0539fc9b97" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 40261 } }
I20260925 22:09:04.465482 23685 leader_election.cc:288] T 68c14fe5479b41b4a73f63844308d4e5 P 9903e7e451bd43eeb861f8d13cfcecfd [CANDIDATE]: Term 1 election: Requested vote from peers ba8d12ca459449179b42f0886b2993bd (127.5.0.66:35773), c2caed2e06a7409c94332d0539fc9b97 (127.5.0.67:40261)
I20260925 22:09:04.465628 23618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68c14fe5479b41b4a73f63844308d4e5" candidate_uuid: "9903e7e451bd43eeb861f8d13cfcecfd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2caed2e06a7409c94332d0539fc9b97"
I20260925 22:09:04.465665 23484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68c14fe5479b41b4a73f63844308d4e5" candidate_uuid: "9903e7e451bd43eeb861f8d13cfcecfd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba8d12ca459449179b42f0886b2993bd"
I20260925 22:09:04.465708 23618 raft_consensus.cc:3037] T 68c14fe5479b41b4a73f63844308d4e5 P c2caed2e06a7409c94332d0539fc9b97 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:04.465728 23484 raft_consensus.cc:3037] T 68c14fe5479b41b4a73f63844308d4e5 P ba8d12ca459449179b42f0886b2993bd [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:04.466629 23484 raft_consensus.cc:2454] T 68c14fe5479b41b4a73f63844308d4e5 P ba8d12ca459449179b42f0886b2993bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9903e7e451bd43eeb861f8d13cfcecfd in term 1.
I20260925 22:09:04.466639 23618 raft_consensus.cc:2454] T 68c14fe5479b41b4a73f63844308d4e5 P c2caed2e06a7409c94332d0539fc9b97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9903e7e451bd43eeb861f8d13cfcecfd in term 1.
I20260925 22:09:04.466890 23286 leader_election.cc:302] T 68c14fe5479b41b4a73f63844308d4e5 P 9903e7e451bd43eeb861f8d13cfcecfd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9903e7e451bd43eeb861f8d13cfcecfd, ba8d12ca459449179b42f0886b2993bd; no voters:
I20260925 22:09:04.466986 23685 raft_consensus.cc:2781] T 68c14fe5479b41b4a73f63844308d4e5 P 9903e7e451bd43eeb861f8d13cfcecfd [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:09:04.467159 23685 raft_consensus.cc:686] T 68c14fe5479b41b4a73f63844308d4e5 P 9903e7e451bd43eeb861f8d13cfcecfd [term 1 LEADER]: Becoming Leader. State: Replica: 9903e7e451bd43eeb861f8d13cfcecfd, State: Running, Role: LEADER
I20260925 22:09:04.467259 23685 consensus_queue.cc:237] T 68c14fe5479b41b4a73f63844308d4e5 P 9903e7e451bd43eeb861f8d13cfcecfd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9903e7e451bd43eeb861f8d13cfcecfd" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 40041 } } peers { permanent_uuid: "ba8d12ca459449179b42f0886b2993bd" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 35773 } } peers { permanent_uuid: "c2caed2e06a7409c94332d0539fc9b97" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 40261 } }
I20260925 22:09:04.468039 23204 catalog_manager.cc:5495] T 68c14fe5479b41b4a73f63844308d4e5 P 9903e7e451bd43eeb861f8d13cfcecfd reported cstate change: term changed from 0 to 1, leader changed from <none> to 9903e7e451bd43eeb861f8d13cfcecfd (127.5.0.65). New cstate: current_term: 1 leader_uuid: "9903e7e451bd43eeb861f8d13cfcecfd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9903e7e451bd43eeb861f8d13cfcecfd" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 40041 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba8d12ca459449179b42f0886b2993bd" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 35773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2caed2e06a7409c94332d0539fc9b97" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 40261 } health_report { overall_health: UNKNOWN } } }
W20260925 22:09:04.478135 23531 tablet.cc:2369] T 68c14fe5479b41b4a73f63844308d4e5 P ba8d12ca459449179b42f0886b2993bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:09:04.487810 23676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:04.485419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44664 (local address 127.5.0.65:40041)
0925 22:09:04.485479 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:04.485481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:04.485491 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:09:04.485553 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:04.485554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:04.485571 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:04.485630 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:09:04.485739 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.486233 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.487106 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.487314 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.487373 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:04.487513 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:09:04.487594 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:09:04.487709 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:04.487748 (+ 39us) server_negotiation.cc:299] Negotiation successful
0925 22:09:04.487756 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 22:09:04.489670 23484 raft_consensus.cc:1261] T 68c14fe5479b41b4a73f63844308d4e5 P ba8d12ca459449179b42f0886b2993bd [term 1 FOLLOWER]: Refusing update from remote peer 9903e7e451bd43eeb861f8d13cfcecfd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:09:04.489670 23618 raft_consensus.cc:1261] T 68c14fe5479b41b4a73f63844308d4e5 P c2caed2e06a7409c94332d0539fc9b97 [term 1 FOLLOWER]: Refusing update from remote peer 9903e7e451bd43eeb861f8d13cfcecfd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:09:04.490002 23688 consensus_queue.cc:1035] T 68c14fe5479b41b4a73f63844308d4e5 P 9903e7e451bd43eeb861f8d13cfcecfd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba8d12ca459449179b42f0886b2993bd" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 35773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:09:04.490159 23685 consensus_queue.cc:1035] T 68c14fe5479b41b4a73f63844308d4e5 P 9903e7e451bd43eeb861f8d13cfcecfd [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2caed2e06a7409c94332d0539fc9b97" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 40261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:09:04.492439 23691 mvcc.cc:205] Tried to move back new op lower bound from 7333372495826755584 to 7333372495738331136. Current Snapshot: MvccSnapshot[applied={T|T < 7333372495826755584}]
I20260925 22:09:04.493249 23693 mvcc.cc:205] Tried to move back new op lower bound from 7333372495826755584 to 7333372495738331136. Current Snapshot: MvccSnapshot[applied={T|T < 7333372495826755584}]
I20260925 22:09:04.493711 23692 mvcc.cc:205] Tried to move back new op lower bound from 7333372495826755584 to 7333372495738331136. Current Snapshot: MvccSnapshot[applied={T|T < 7333372495826755584}]
I20260925 22:09:04.494701 23204 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:09:04.494789 23204 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:09:04.495496 23204 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aac373204e5d4cd4a598e9bf2b2ff7a4: Sending DeleteTablet for 3 replicas of tablet 68c14fe5479b41b4a73f63844308d4e5
I20260925 22:09:04.495854 23464 tablet_service.cc:1514] Processing DeleteTablet for tablet 68c14fe5479b41b4a73f63844308d4e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:09:04 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:37902
I20260925 22:09:04.495848 23330 tablet_service.cc:1514] Processing DeleteTablet for tablet 68c14fe5479b41b4a73f63844308d4e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:09:04 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:44650
I20260925 22:09:04.496091 23702 tablet_replica.cc:330] T 68c14fe5479b41b4a73f63844308d4e5 P 9903e7e451bd43eeb861f8d13cfcecfd: stopping tablet replica
I20260925 22:09:04.496168 23598 tablet_service.cc:1514] Processing DeleteTablet for tablet 68c14fe5479b41b4a73f63844308d4e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:09:04 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:51830
I20260925 22:09:04.496197 23702 raft_consensus.cc:2229] T 68c14fe5479b41b4a73f63844308d4e5 P 9903e7e451bd43eeb861f8d13cfcecfd [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:09:04.496295 23702 raft_consensus.cc:2258] T 68c14fe5479b41b4a73f63844308d4e5 P 9903e7e451bd43eeb861f8d13cfcecfd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:09:04.496413 23703 tablet_replica.cc:330] T 68c14fe5479b41b4a73f63844308d4e5 P c2caed2e06a7409c94332d0539fc9b97: stopping tablet replica
I20260925 22:09:04.496575 23703 raft_consensus.cc:2229] T 68c14fe5479b41b4a73f63844308d4e5 P c2caed2e06a7409c94332d0539fc9b97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:09:04.496626 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 23267
I20260925 22:09:04.496764 23703 raft_consensus.cc:2258] T 68c14fe5479b41b4a73f63844308d4e5 P c2caed2e06a7409c94332d0539fc9b97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:09:04.497774 23703 ts_tablet_manager.cc:1905] T 68c14fe5479b41b4a73f63844308d4e5 P c2caed2e06a7409c94332d0539fc9b97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:09:04.499379 23703 ts_tablet_manager.cc:1918] T 68c14fe5479b41b4a73f63844308d4e5 P c2caed2e06a7409c94332d0539fc9b97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:09:04.499444 23703 log.cc:1196] T 68c14fe5479b41b4a73f63844308d4e5 P c2caed2e06a7409c94332d0539fc9b97: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-2/wal/wals/68c14fe5479b41b4a73f63844308d4e5
I20260925 22:09:04.499702 23703 ts_tablet_manager.cc:1939] T 68c14fe5479b41b4a73f63844308d4e5 P c2caed2e06a7409c94332d0539fc9b97: Deleting consensus metadata
I20260925 22:09:04.496085 23701 tablet_replica.cc:330] T 68c14fe5479b41b4a73f63844308d4e5 P ba8d12ca459449179b42f0886b2993bd: stopping tablet replica
I20260925 22:09:04.500164 23189 catalog_manager.cc:4841] TS c2caed2e06a7409c94332d0539fc9b97 (127.5.0.67:40261): tablet 68c14fe5479b41b4a73f63844308d4e5 (table test-table [id=f1a1498e1bae45d59a61c242188f5548]) successfully deleted
I20260925 22:09:04.500269 23701 raft_consensus.cc:2229] T 68c14fe5479b41b4a73f63844308d4e5 P ba8d12ca459449179b42f0886b2993bd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:09:04.500401 23701 raft_consensus.cc:2258] T 68c14fe5479b41b4a73f63844308d4e5 P ba8d12ca459449179b42f0886b2993bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:09:04.500804 23701 ts_tablet_manager.cc:1905] T 68c14fe5479b41b4a73f63844308d4e5 P ba8d12ca459449179b42f0886b2993bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:09:04.502249 23701 ts_tablet_manager.cc:1918] T 68c14fe5479b41b4a73f63844308d4e5 P ba8d12ca459449179b42f0886b2993bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:09:04.502307 23701 log.cc:1196] T 68c14fe5479b41b4a73f63844308d4e5 P ba8d12ca459449179b42f0886b2993bd: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373908987421-5121-0/minicluster-data/ts-1/wal/wals/68c14fe5479b41b4a73f63844308d4e5
I20260925 22:09:04.502626 23701 ts_tablet_manager.cc:1939] T 68c14fe5479b41b4a73f63844308d4e5 P ba8d12ca459449179b42f0886b2993bd: Deleting consensus metadata
W20260925 22:09:04.502671 23192 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.0.65:40041 (error 108)
I20260925 22:09:04.502928 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 23401
I20260925 22:09:04.503072 23192 catalog_manager.cc:4841] TS ba8d12ca459449179b42f0886b2993bd (127.5.0.66:35773): tablet 68c14fe5479b41b4a73f63844308d4e5 (table test-table [id=f1a1498e1bae45d59a61c242188f5548]) successfully deleted
I20260925 22:09:04.503539 23674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:04.503023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:40041 (local address 127.0.0.1:44678)
0925 22:09:04.503455 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:04.503499 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.65:40041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411}
W20260925 22:09:04.503664 23192 catalog_manager.cc:4584] TS 9903e7e451bd43eeb861f8d13cfcecfd (127.5.0.65:40041): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 68c14fe5479b41b4a73f63844308d4e5: Network error: Client connection negotiation failed: client connection to 127.5.0.65:40041: connect: Connection refused (error 111)
I20260925 22:09:04.507586 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 23535
I20260925 22:09:04.511993 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 23173
2026-09-25T22:09:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1910 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-25T22:09:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:09:04Z Disabled control of system clock
I20260925 22:09:04.539191 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:34807
--webserver_interface=127.5.0.126
--webserver_port=0
--builtin_ntp_servers=127.5.0.84:34549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:34807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 22:09:04.619422 23708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:04.619848 23708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:04.619875 23708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:04.621166 23708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:09:04.621209 23708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:04.621222 23708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:04.621235 23708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:09:04.621248 23708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:09:04.622645 23708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:34549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:34807
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:34807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:04.622905 23708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:04.623204 23708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:04.628207 23716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:04.628250 23713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:04.628433 23708 server_base.cc:1034] running on GCE node
W20260925 22:09:04.628213 23714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:04.628753 23708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:04.628998 23708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:04.630182 23708 hybrid_clock.cc:648] HybridClock initialized: now 1790374144630161 us; error 40 us; skew 500 ppm
I20260925 22:09:04.631335 23708 webserver.cc:460] Webserver started at http://127.5.0.126:38487/ using document root <none> and password file <none>
I20260925 22:09:04.631608 23708 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:04.631654 23708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:04.631747 23708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:04.632894 23708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "c38b71603c944e71a2f149b5b496e09e"
format_stamp: "Formatted at 2026-09-25 22:09:04 on dist-test-slave-7mqr"
server_key: "82e6e0ec1a08622276f0ab2fbd417e06"
server_key_iv: "36e58ea383c06d90a620d5f03a7e1a8e"
server_key_version: "encryptionkey@0"
I20260925 22:09:04.633256 23708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "c38b71603c944e71a2f149b5b496e09e"
format_stamp: "Formatted at 2026-09-25 22:09:04 on dist-test-slave-7mqr"
server_key: "82e6e0ec1a08622276f0ab2fbd417e06"
server_key_iv: "36e58ea383c06d90a620d5f03a7e1a8e"
server_key_version: "encryptionkey@0"
I20260925 22:09:04.634471 23708 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:09:04.635141 23722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:04.635344 23708 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:09:04.635434 23708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "c38b71603c944e71a2f149b5b496e09e"
format_stamp: "Formatted at 2026-09-25 22:09:04 on dist-test-slave-7mqr"
server_key: "82e6e0ec1a08622276f0ab2fbd417e06"
server_key_iv: "36e58ea383c06d90a620d5f03a7e1a8e"
server_key_version: "encryptionkey@0"
I20260925 22:09:04.635489 23708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:04.645141 23708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:04.645392 23708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:04.645493 23708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:04.649449 23708 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:34807
I20260925 22:09:04.649482 23774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:34807 every 8 connection(s)
I20260925 22:09:04.649819 23708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:09:04.650727 23775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:04.652999 23775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c38b71603c944e71a2f149b5b496e09e: Bootstrap starting.
I20260925 22:09:04.653427 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 23708
I20260925 22:09:04.653543 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:09:04.653617 23775 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c38b71603c944e71a2f149b5b496e09e: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:04.653752 5121 external_mini_cluster.cc:1428] Setting key a8cccac6302248085cda8105976b542c
I20260925 22:09:04.653977 23775 log.cc:824] T 00000000000000000000000000000000 P c38b71603c944e71a2f149b5b496e09e: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:04.654685 23775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c38b71603c944e71a2f149b5b496e09e: No bootstrap required, opened a new log
I20260925 22:09:04.656888 23775 raft_consensus.cc:348] T 00000000000000000000000000000000 P c38b71603c944e71a2f149b5b496e09e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c38b71603c944e71a2f149b5b496e09e" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34807 } }
I20260925 22:09:04.657050 23775 raft_consensus.cc:374] T 00000000000000000000000000000000 P c38b71603c944e71a2f149b5b496e09e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:04.657083 23775 raft_consensus.cc:729] T 00000000000000000000000000000000 P c38b71603c944e71a2f149b5b496e09e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c38b71603c944e71a2f149b5b496e09e, State: Initialized, Role: FOLLOWER
I20260925 22:09:04.657193 23775 consensus_queue.cc:260] T 00000000000000000000000000000000 P c38b71603c944e71a2f149b5b496e09e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c38b71603c944e71a2f149b5b496e09e" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34807 } }
I20260925 22:09:04.657258 23775 raft_consensus.cc:388] T 00000000000000000000000000000000 P c38b71603c944e71a2f149b5b496e09e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:09:04.657294 23775 raft_consensus.cc:482] T 00000000000000000000000000000000 P c38b71603c944e71a2f149b5b496e09e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:09:04.657339 23775 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c38b71603c944e71a2f149b5b496e09e [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:04.657860 23775 raft_consensus.cc:504] T 00000000000000000000000000000000 P c38b71603c944e71a2f149b5b496e09e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c38b71603c944e71a2f149b5b496e09e" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34807 } }
I20260925 22:09:04.657982 23775 leader_election.cc:302] T 00000000000000000000000000000000 P c38b71603c944e71a2f149b5b496e09e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c38b71603c944e71a2f149b5b496e09e; no voters:
I20260925 22:09:04.658169 23775 leader_election.cc:288] T 00000000000000000000000000000000 P c38b71603c944e71a2f149b5b496e09e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:09:04.658259 23780 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c38b71603c944e71a2f149b5b496e09e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:09:04.658442 23780 raft_consensus.cc:686] T 00000000000000000000000000000000 P c38b71603c944e71a2f149b5b496e09e [term 1 LEADER]: Becoming Leader. State: Replica: c38b71603c944e71a2f149b5b496e09e, State: Running, Role: LEADER
I20260925 22:09:04.658501 23775 sys_catalog.cc:564] T 00000000000000000000000000000000 P c38b71603c944e71a2f149b5b496e09e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:09:04.658628 23780 consensus_queue.cc:237] T 00000000000000000000000000000000 P c38b71603c944e71a2f149b5b496e09e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c38b71603c944e71a2f149b5b496e09e" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34807 } }
I20260925 22:09:04.659241 23782 sys_catalog.cc:455] T 00000000000000000000000000000000 P c38b71603c944e71a2f149b5b496e09e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c38b71603c944e71a2f149b5b496e09e. Latest consensus state: current_term: 1 leader_uuid: "c38b71603c944e71a2f149b5b496e09e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c38b71603c944e71a2f149b5b496e09e" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34807 } } }
I20260925 22:09:04.659327 23782 sys_catalog.cc:458] T 00000000000000000000000000000000 P c38b71603c944e71a2f149b5b496e09e [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:04.659479 23781 sys_catalog.cc:455] T 00000000000000000000000000000000 P c38b71603c944e71a2f149b5b496e09e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c38b71603c944e71a2f149b5b496e09e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c38b71603c944e71a2f149b5b496e09e" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 34807 } } }
I20260925 22:09:04.659539 23781 sys_catalog.cc:458] T 00000000000000000000000000000000 P c38b71603c944e71a2f149b5b496e09e [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:04.659808 23779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:04.654313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37648 (local address 127.5.0.126:34807)
0925 22:09:04.654502 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:04.654516 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:04.654541 (+ 25us) server_negotiation.cc:407] Connection header received
0925 22:09:04.654971 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:04.654980 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:04.655037 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:04.655147 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:04.655389 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.656305 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.657444 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.657958 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.658114 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:04.658839 (+ 725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:04.658843 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:04.658843 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:04.659062 (+ 219us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:04.659104 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:04.659118 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:04.659181 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:04.659463 (+ 282us) server_negotiation.cc:299] Negotiation successful
0925 22:09:04.659552 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":79,"threads_started":1}
W20260925 22:09:04.660351 23796 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c38b71603c944e71a2f149b5b496e09e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:09:04.660403 23796 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:09:04.660487 23789 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:09:04.660662 23789 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:09:04.662099 23789 catalog_manager.cc:1347] Generated new cluster ID: bf19a38eef1e4a06a41890d7faae2174
I20260925 22:09:04.662153 23789 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:09:04.673188 23789 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:09:04.673898 23789 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:09:04.683938 23789 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c38b71603c944e71a2f149b5b496e09e: Generated new TSK 0
I20260925 22:09:04.684161 23789 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:09:04.690357 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.65:0
--local_ip_for_outbound_sockets=127.5.0.65
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:34807
--builtin_ntp_servers=127.5.0.84:34549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 22:09:04.780586 23799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:04.780820 23799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:04.780857 23799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:04.782001 23799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:04.782063 23799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:04.782107 23799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.65
I20260925 22:09:04.783454 23799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:34549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:34807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.65
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:04.783715 23799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:04.784072 23799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:04.789109 23807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:04.789101 23805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:04.789100 23804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:04.789323 23799 server_base.cc:1034] running on GCE node
I20260925 22:09:04.789517 23799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:04.789711 23799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:04.790860 23799 hybrid_clock.cc:648] HybridClock initialized: now 1790374144790849 us; error 30 us; skew 500 ppm
I20260925 22:09:04.792198 23799 webserver.cc:460] Webserver started at http://127.5.0.65:35973/ using document root <none> and password file <none>
I20260925 22:09:04.792398 23799 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:04.792486 23799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:04.792578 23799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:04.793592 23799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-0/data/instance:
uuid: "08873ffcfc304d1880333e420f583a1c"
format_stamp: "Formatted at 2026-09-25 22:09:04 on dist-test-slave-7mqr"
server_key: "9e758edcfceb54bd66f0845eb28b6d2d"
server_key_iv: "0b68aad188d66c42007343f8e78e014b"
server_key_version: "encryptionkey@0"
I20260925 22:09:04.793895 23799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance:
uuid: "08873ffcfc304d1880333e420f583a1c"
format_stamp: "Formatted at 2026-09-25 22:09:04 on dist-test-slave-7mqr"
server_key: "9e758edcfceb54bd66f0845eb28b6d2d"
server_key_iv: "0b68aad188d66c42007343f8e78e014b"
server_key_version: "encryptionkey@0"
I20260925 22:09:04.795152 23799 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:09:04.796089 23813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:04.796299 23799 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:09:04.796411 23799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-0/wal
uuid: "08873ffcfc304d1880333e420f583a1c"
format_stamp: "Formatted at 2026-09-25 22:09:04 on dist-test-slave-7mqr"
server_key: "9e758edcfceb54bd66f0845eb28b6d2d"
server_key_iv: "0b68aad188d66c42007343f8e78e014b"
server_key_version: "encryptionkey@0"
I20260925 22:09:04.796533 23799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:04.812014 23799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:04.812327 23799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:04.812489 23799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:04.812724 23799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:09:04.813051 23799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:09:04.813104 23799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:04.813145 23799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:09:04.813171 23799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:04.818720 23799 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.65:44551
I20260925 22:09:04.818794 23926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.65:44551 every 8 connection(s)
I20260925 22:09:04.819131 23799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
I20260925 22:09:04.824460 23779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:04.819868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:45963 (local address 127.5.0.126:34807)
0925 22:09:04.819986 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:04.819988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:04.820216 (+ 228us) server_negotiation.cc:407] Connection header received
0925 22:09:04.820747 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:04.820748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:04.820761 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:04.820817 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:04.821501 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.822015 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.823009 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.823224 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.823305 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:04.824173 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:04.824174 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:04.824175 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:04.824179 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:04.824196 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:04.824200 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:04.824254 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:04.824350 (+ 96us) server_negotiation.cc:299] Negotiation successful
0925 22:09:04.824379 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260925 22:09:04.824683 23929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:04.819849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34807 (local address 127.5.0.65:45963)
0925 22:09:04.820090 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:04.820124 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:04.820640 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:04.820890 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:04.820894 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:04.820909 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:04.821404 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:04.821408 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.822117 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.822119 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:04.822915 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:04.822917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.823315 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.823316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:04.823454 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:04.824033 (+ 579us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:09:04.824033 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:04.824061 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:09:04.824084 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:04.824259 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:04.824264 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:04.824267 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:09:04.824351 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 22:09:04.824454 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":122,"threads_started":1}
I20260925 22:09:04.825205 23927 heartbeater.cc:347] Connected to a master server at 127.5.0.126:34807
I20260925 22:09:04.825282 23927 heartbeater.cc:464] Registering TS with master...
I20260925 22:09:04.825476 23927 heartbeater.cc:511] Master 127.5.0.126:34807 requested a full tablet report, sending...
I20260925 22:09:04.825877 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 23799
I20260925 22:09:04.826014 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance
I20260925 22:09:04.826050 23739 ts_manager.cc:194] Registered new tserver with Master: 08873ffcfc304d1880333e420f583a1c (127.5.0.65:44551)
I20260925 22:09:04.826249 5121 external_mini_cluster.cc:1428] Setting key b45fa4f6d6c17e974cdaae7498a14707
I20260925 22:09:04.826829 23739 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.0.65:45963
I20260925 22:09:04.827289 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.66:0
--local_ip_for_outbound_sockets=127.5.0.66
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:34807
--builtin_ntp_servers=127.5.0.84:34549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 22:09:04.910300 23930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:04.910533 23930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:04.910557 23930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:04.911727 23930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:04.911768 23930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:04.911795 23930 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.66
I20260925 22:09:04.913167 23930 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:34549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:34807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.66
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:04.913417 23930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:04.913743 23930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:04.918660 23935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:04.918668 23936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:04.918659 23938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:04.918857 23930 server_base.cc:1034] running on GCE node
I20260925 22:09:04.919183 23930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:04.919437 23930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:04.920598 23930 hybrid_clock.cc:648] HybridClock initialized: now 1790374144920594 us; error 36 us; skew 500 ppm
I20260925 22:09:04.921880 23930 webserver.cc:460] Webserver started at http://127.5.0.66:35799/ using document root <none> and password file <none>
I20260925 22:09:04.922139 23930 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:04.922209 23930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:04.922358 23930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:04.923413 23930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-1/data/instance:
uuid: "f207febf699b41558ee44f228971f2d3"
format_stamp: "Formatted at 2026-09-25 22:09:04 on dist-test-slave-7mqr"
server_key: "fde4a67345084f515f5c12ec7fa0e68f"
server_key_iv: "b9e0e1050ceed35f8214e3ac39cffa07"
server_key_version: "encryptionkey@0"
I20260925 22:09:04.923768 23930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance:
uuid: "f207febf699b41558ee44f228971f2d3"
format_stamp: "Formatted at 2026-09-25 22:09:04 on dist-test-slave-7mqr"
server_key: "fde4a67345084f515f5c12ec7fa0e68f"
server_key_iv: "b9e0e1050ceed35f8214e3ac39cffa07"
server_key_version: "encryptionkey@0"
I20260925 22:09:04.925051 23930 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:09:04.925774 23944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:04.925942 23930 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:09:04.926038 23930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-1/wal
uuid: "f207febf699b41558ee44f228971f2d3"
format_stamp: "Formatted at 2026-09-25 22:09:04 on dist-test-slave-7mqr"
server_key: "fde4a67345084f515f5c12ec7fa0e68f"
server_key_iv: "b9e0e1050ceed35f8214e3ac39cffa07"
server_key_version: "encryptionkey@0"
I20260925 22:09:04.926124 23930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:04.942775 23930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:04.943090 23930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:04.943245 23930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:04.943478 23930 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:09:04.943809 23930 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:09:04.943861 23930 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:04.943897 23930 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:09:04.943998 23930 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:04.949630 23930 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.66:39195
I20260925 22:09:04.949712 24057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.66:39195 every 8 connection(s)
I20260925 22:09:04.950037 23930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
I20260925 22:09:04.952184 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 23930
I20260925 22:09:04.952282 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance
I20260925 22:09:04.952541 5121 external_mini_cluster.cc:1428] Setting key d7ce8c596f22657b757638c6558acca5
I20260925 22:09:04.953466 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.67:0
--local_ip_for_outbound_sockets=127.5.0.67
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:34807
--builtin_ntp_servers=127.5.0.84:34549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 22:09:04.955693 23779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:04.950778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:39287 (local address 127.5.0.126:34807)
0925 22:09:04.950824 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:04.950826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:04.951083 (+ 257us) server_negotiation.cc:407] Connection header received
0925 22:09:04.951565 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:04.951566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:04.951577 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:04.951637 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:04.952354 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.953034 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.954158 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.954468 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.954545 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:04.955445 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:04.955446 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:04.955446 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:04.955451 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:04.955468 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:04.955471 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:04.955516 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:04.955614 (+ 98us) server_negotiation.cc:299] Negotiation successful
0925 22:09:04.955636 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 22:09:04.956017 24060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:04.950751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34807 (local address 127.5.0.66:39287)
0925 22:09:04.950921 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:04.950954 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:04.951445 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:04.951691 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:04.951694 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:04.951702 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:04.952220 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:04.952224 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.953151 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.953152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:04.954071 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:04.954073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:04.954558 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:04.954558 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:04.954704 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:04.955283 (+ 579us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:09:04.955284 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:04.955325 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:09:04.955354 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:04.955537 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:04.955541 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:04.955544 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:09:04.955611 (+ 67us) client_negotiation.cc:240] Negotiation successful
0925 22:09:04.955726 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
I20260925 22:09:04.956511 24058 heartbeater.cc:347] Connected to a master server at 127.5.0.126:34807
I20260925 22:09:04.956609 24058 heartbeater.cc:464] Registering TS with master...
I20260925 22:09:04.956799 24058 heartbeater.cc:511] Master 127.5.0.126:34807 requested a full tablet report, sending...
I20260925 22:09:04.957168 23739 ts_manager.cc:194] Registered new tserver with Master: f207febf699b41558ee44f228971f2d3 (127.5.0.66:39195)
I20260925 22:09:04.957716 23739 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.0.66:39287
W20260925 22:09:05.035211 24061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:05.035408 24061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:05.035430 24061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:05.036593 24061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:05.036634 24061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:05.036659 24061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.67
I20260925 22:09:05.038064 24061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:34549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:34807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.67
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:05.038322 24061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:05.038659 24061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:05.043814 24069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:05.043831 24067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:05.043962 24061 server_base.cc:1034] running on GCE node
W20260925 22:09:05.043833 24066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:05.044284 24061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:05.044549 24061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:05.045761 24061 hybrid_clock.cc:648] HybridClock initialized: now 1790374145045739 us; error 39 us; skew 500 ppm
I20260925 22:09:05.054090 24061 webserver.cc:460] Webserver started at http://127.5.0.67:40901/ using document root <none> and password file <none>
I20260925 22:09:05.054380 24061 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:05.054435 24061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:05.054605 24061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:05.055712 24061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-2/data/instance:
uuid: "ef21fd7d25da41c6bc494deaffbc77e2"
format_stamp: "Formatted at 2026-09-25 22:09:05 on dist-test-slave-7mqr"
server_key: "35aa80d4b7277284f432943019fe272e"
server_key_iv: "2d6f331fa447f3334a69f99947834e1a"
server_key_version: "encryptionkey@0"
I20260925 22:09:05.056136 24061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance:
uuid: "ef21fd7d25da41c6bc494deaffbc77e2"
format_stamp: "Formatted at 2026-09-25 22:09:05 on dist-test-slave-7mqr"
server_key: "35aa80d4b7277284f432943019fe272e"
server_key_iv: "2d6f331fa447f3334a69f99947834e1a"
server_key_version: "encryptionkey@0"
I20260925 22:09:05.057466 24061 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:09:05.058254 24075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:05.058459 24061 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:09:05.058565 24061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-2/wal
uuid: "ef21fd7d25da41c6bc494deaffbc77e2"
format_stamp: "Formatted at 2026-09-25 22:09:05 on dist-test-slave-7mqr"
server_key: "35aa80d4b7277284f432943019fe272e"
server_key_iv: "2d6f331fa447f3334a69f99947834e1a"
server_key_version: "encryptionkey@0"
I20260925 22:09:05.058633 24061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:05.066847 24061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:05.067080 24061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:05.067178 24061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:05.067329 24061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:09:05.067575 24061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:09:05.067605 24061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:05.067624 24061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:09:05.067637 24061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:05.073544 24061 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.67:38955
I20260925 22:09:05.073589 24188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.67:38955 every 8 connection(s)
I20260925 22:09:05.073909 24061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
I20260925 22:09:05.078853 23779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:05.074649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:35747 (local address 127.5.0.126:34807)
0925 22:09:05.074726 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:05.074729 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:05.074855 (+ 126us) server_negotiation.cc:407] Connection header received
0925 22:09:05.075326 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:05.075328 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:05.075337 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:05.075389 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:05.076118 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:05.076627 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:05.077567 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:05.077755 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:05.077818 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:05.078576 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:05.078578 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:05.078578 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:05.078582 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:05.078597 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:05.078600 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:05.078656 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:05.078773 (+ 117us) server_negotiation.cc:299] Negotiation successful
0925 22:09:05.078794 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 22:09:05.078890 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 24061
I20260925 22:09:05.078998 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance
I20260925 22:09:05.079156 24191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:05.074568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:34807 (local address 127.5.0.67:35747)
0925 22:09:05.074742 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:05.074775 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:05.075247 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:05.075441 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:05.075444 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:05.075452 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:05.076017 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:05.076023 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:05.076710 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:05.076711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:05.077485 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:05.077487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:05.077831 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:05.077832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:05.077939 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:05.078425 (+ 486us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:09:05.078426 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:05.078453 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:09:05.078474 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:05.078686 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:05.078692 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:05.078696 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:09:05.078771 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 22:09:05.078885 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":68,"threads_started":1}
I20260925 22:09:05.079227 5121 external_mini_cluster.cc:1428] Setting key 1f80aafe9d0d58aede18be1a33d40d04
I20260925 22:09:05.079628 24189 heartbeater.cc:347] Connected to a master server at 127.5.0.126:34807
I20260925 22:09:05.079730 24189 heartbeater.cc:464] Registering TS with master...
I20260925 22:09:05.079949 24189 heartbeater.cc:511] Master 127.5.0.126:34807 requested a full tablet report, sending...
I20260925 22:09:05.080351 23739 ts_manager.cc:194] Registered new tserver with Master: ef21fd7d25da41c6bc494deaffbc77e2 (127.5.0.67:38955)
I20260925 22:09:05.080680 23739 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.0.67:35747
I20260925 22:09:05.091032 5121 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:09:05.095541 23779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:05.092428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37658 (local address 127.5.0.126:34807)
0925 22:09:05.092509 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:05.092512 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:05.092619 (+ 107us) server_negotiation.cc:407] Connection header received
0925 22:09:05.092714 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:05.092715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:05.092726 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:05.092792 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:05.092970 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:05.093453 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:05.094241 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:05.094436 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:05.094510 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:05.095242 (+ 732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:05.095243 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:05.095243 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:05.095248 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:05.095261 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:05.095265 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:05.095367 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:05.095437 (+ 70us) server_negotiation.cc:299] Negotiation successful
0925 22:09:05.095447 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 22:09:05.097018 23739 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37658:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:09:05.098588 23739 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:09:05.105109 24201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:05.100960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38014 (local address 127.5.0.66:39195)
0925 22:09:05.101147 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:05.101151 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:05.101584 (+ 433us) server_negotiation.cc:407] Connection header received
0925 22:09:05.101628 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:05.101630 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:05.101645 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:05.101716 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:09:05.101813 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:05.102475 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:05.103981 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:05.104729 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:05.104801 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:05.104895 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:05.104983 (+ 88us) server_negotiation.cc:299] Negotiation successful
0925 22:09:05.105019 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":85,"threads_started":1}
I20260925 22:09:05.105104 24197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:05.100648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:38955 (local address 127.0.0.1:42466)
0925 22:09:05.100773 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:05.100812 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:05.100870 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:05.101211 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:05.101214 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:05.101216 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:09:05.101282 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:05.101284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:05.102737 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:05.102738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:05.103869 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:05.103872 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:05.104824 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:05.104825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:05.104908 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:05.104913 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:09:05.104952 (+ 39us) client_negotiation.cc:240] Negotiation successful
0925 22:09:05.105025 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":7,"thread_start_us":81,"threads_started":1}
I20260925 22:09:05.105103 24198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:05.100876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:39195 (local address 127.0.0.1:38014)
0925 22:09:05.101462 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:05.101473 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:05.101519 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:05.101713 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:05.101714 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:05.101716 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:09:05.101750 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:05.101752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:05.102577 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:05.102578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:05.103869 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:05.103872 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:05.104812 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:05.104813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:05.104888 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:05.104903 (+ 15us) client_negotiation.cc:769] Sending connection context
0925 22:09:05.104972 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 22:09:05.105016 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"mutex_wait_us":5,"thread_start_us":66,"threads_started":1}
I20260925 22:09:05.105406 24199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:05.100737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42466 (local address 127.5.0.67:38955)
0925 22:09:05.100979 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:05.100984 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:05.100995 (+ 11us) server_negotiation.cc:407] Connection header received
0925 22:09:05.101057 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:05.101059 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:05.101075 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:05.101199 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:09:05.101370 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:05.102653 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:05.103980 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:05.104728 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:05.105151 (+ 423us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:05.105253 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:05.105280 (+ 27us) server_negotiation.cc:299] Negotiation successful
0925 22:09:05.105324 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":86,"threads_started":1}
I20260925 22:09:05.105921 23992 tablet_service.cc:1467] Processing CreateTablet for tablet 9585b43dce534181b11e360d8cca8cc0 (DEFAULT_TABLE table=test-table [id=5198cfc0ad7c41e39a07daddbc7bc233]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:09:05.106065 24123 tablet_service.cc:1467] Processing CreateTablet for tablet 9585b43dce534181b11e360d8cca8cc0 (DEFAULT_TABLE table=test-table [id=5198cfc0ad7c41e39a07daddbc7bc233]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:09:05.106277 24200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:05.100650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:44551 (local address 127.0.0.1:33236)
0925 22:09:05.101138 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:05.101149 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:05.101194 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:05.101373 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:05.101374 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:05.101376 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:09:05.101417 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:05.101419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:05.103991 (+ 2572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:05.103991 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:05.104653 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:05.104654 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:05.106038 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:05.106039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:05.106106 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:05.106110 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:09:05.106171 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 22:09:05.106211 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":90,"threads_started":1}
I20260925 22:09:05.106283 23992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9585b43dce534181b11e360d8cca8cc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:05.106309 24202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:05.101000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33236 (local address 127.5.0.65:44551)
0925 22:09:05.101218 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:05.101221 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:05.101231 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:09:05.101269 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:05.101271 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:05.101285 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:05.101364 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:09:05.101503 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:05.102655 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:05.104741 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:05.105952 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:05.106021 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:05.106111 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:05.106185 (+ 74us) server_negotiation.cc:299] Negotiation successful
0925 22:09:05.106226 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":111,"threads_started":1}
I20260925 22:09:05.106405 24123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9585b43dce534181b11e360d8cca8cc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:05.108755 24203 tablet_bootstrap.cc:492] T 9585b43dce534181b11e360d8cca8cc0 P f207febf699b41558ee44f228971f2d3: Bootstrap starting.
I20260925 22:09:05.109186 24204 tablet_bootstrap.cc:492] T 9585b43dce534181b11e360d8cca8cc0 P ef21fd7d25da41c6bc494deaffbc77e2: Bootstrap starting.
I20260925 22:09:05.109457 24203 tablet_bootstrap.cc:654] T 9585b43dce534181b11e360d8cca8cc0 P f207febf699b41558ee44f228971f2d3: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:05.109480 23861 tablet_service.cc:1467] Processing CreateTablet for tablet 9585b43dce534181b11e360d8cca8cc0 (DEFAULT_TABLE table=test-table [id=5198cfc0ad7c41e39a07daddbc7bc233]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:09:05.109784 24203 log.cc:824] T 9585b43dce534181b11e360d8cca8cc0 P f207febf699b41558ee44f228971f2d3: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:05.109794 23861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9585b43dce534181b11e360d8cca8cc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:05.109818 24204 tablet_bootstrap.cc:654] T 9585b43dce534181b11e360d8cca8cc0 P ef21fd7d25da41c6bc494deaffbc77e2: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:05.110119 24204 log.cc:824] T 9585b43dce534181b11e360d8cca8cc0 P ef21fd7d25da41c6bc494deaffbc77e2: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:05.110553 24203 tablet_bootstrap.cc:492] T 9585b43dce534181b11e360d8cca8cc0 P f207febf699b41558ee44f228971f2d3: No bootstrap required, opened a new log
I20260925 22:09:05.110621 24203 ts_tablet_manager.cc:1397] T 9585b43dce534181b11e360d8cca8cc0 P f207febf699b41558ee44f228971f2d3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:09:05.110862 24204 tablet_bootstrap.cc:492] T 9585b43dce534181b11e360d8cca8cc0 P ef21fd7d25da41c6bc494deaffbc77e2: No bootstrap required, opened a new log
I20260925 22:09:05.110934 24204 ts_tablet_manager.cc:1397] T 9585b43dce534181b11e360d8cca8cc0 P ef21fd7d25da41c6bc494deaffbc77e2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:09:05.112700 24203 raft_consensus.cc:348] T 9585b43dce534181b11e360d8cca8cc0 P f207febf699b41558ee44f228971f2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef21fd7d25da41c6bc494deaffbc77e2" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 38955 } } peers { permanent_uuid: "08873ffcfc304d1880333e420f583a1c" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 44551 } } peers { permanent_uuid: "f207febf699b41558ee44f228971f2d3" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 39195 } }
I20260925 22:09:05.112851 24203 raft_consensus.cc:374] T 9585b43dce534181b11e360d8cca8cc0 P f207febf699b41558ee44f228971f2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:05.112895 24203 raft_consensus.cc:729] T 9585b43dce534181b11e360d8cca8cc0 P f207febf699b41558ee44f228971f2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f207febf699b41558ee44f228971f2d3, State: Initialized, Role: FOLLOWER
I20260925 22:09:05.113006 24203 consensus_queue.cc:260] T 9585b43dce534181b11e360d8cca8cc0 P f207febf699b41558ee44f228971f2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef21fd7d25da41c6bc494deaffbc77e2" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 38955 } } peers { permanent_uuid: "08873ffcfc304d1880333e420f583a1c" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 44551 } } peers { permanent_uuid: "f207febf699b41558ee44f228971f2d3" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 39195 } }
I20260925 22:09:05.113283 24203 ts_tablet_manager.cc:1428] T 9585b43dce534181b11e360d8cca8cc0 P f207febf699b41558ee44f228971f2d3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 22:09:05.113293 24207 tablet_bootstrap.cc:492] T 9585b43dce534181b11e360d8cca8cc0 P 08873ffcfc304d1880333e420f583a1c: Bootstrap starting.
I20260925 22:09:05.113436 24058 heartbeater.cc:503] Master 127.5.0.126:34807 was elected leader, sending a full tablet report...
I20260925 22:09:05.113581 24204 raft_consensus.cc:348] T 9585b43dce534181b11e360d8cca8cc0 P ef21fd7d25da41c6bc494deaffbc77e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef21fd7d25da41c6bc494deaffbc77e2" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 38955 } } peers { permanent_uuid: "08873ffcfc304d1880333e420f583a1c" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 44551 } } peers { permanent_uuid: "f207febf699b41558ee44f228971f2d3" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 39195 } }
I20260925 22:09:05.113725 24204 raft_consensus.cc:374] T 9585b43dce534181b11e360d8cca8cc0 P ef21fd7d25da41c6bc494deaffbc77e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:05.113762 24204 raft_consensus.cc:729] T 9585b43dce534181b11e360d8cca8cc0 P ef21fd7d25da41c6bc494deaffbc77e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef21fd7d25da41c6bc494deaffbc77e2, State: Initialized, Role: FOLLOWER
I20260925 22:09:05.113870 24204 consensus_queue.cc:260] T 9585b43dce534181b11e360d8cca8cc0 P ef21fd7d25da41c6bc494deaffbc77e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef21fd7d25da41c6bc494deaffbc77e2" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 38955 } } peers { permanent_uuid: "08873ffcfc304d1880333e420f583a1c" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 44551 } } peers { permanent_uuid: "f207febf699b41558ee44f228971f2d3" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 39195 } }
I20260925 22:09:05.113996 24207 tablet_bootstrap.cc:654] T 9585b43dce534181b11e360d8cca8cc0 P 08873ffcfc304d1880333e420f583a1c: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:05.114117 24204 ts_tablet_manager.cc:1428] T 9585b43dce534181b11e360d8cca8cc0 P ef21fd7d25da41c6bc494deaffbc77e2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 22:09:05.114185 24189 heartbeater.cc:503] Master 127.5.0.126:34807 was elected leader, sending a full tablet report...
I20260925 22:09:05.114562 24207 log.cc:824] T 9585b43dce534181b11e360d8cca8cc0 P 08873ffcfc304d1880333e420f583a1c: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:05.115399 24207 tablet_bootstrap.cc:492] T 9585b43dce534181b11e360d8cca8cc0 P 08873ffcfc304d1880333e420f583a1c: No bootstrap required, opened a new log
I20260925 22:09:05.115480 24207 ts_tablet_manager.cc:1397] T 9585b43dce534181b11e360d8cca8cc0 P 08873ffcfc304d1880333e420f583a1c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:09:05.117466 24207 raft_consensus.cc:348] T 9585b43dce534181b11e360d8cca8cc0 P 08873ffcfc304d1880333e420f583a1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef21fd7d25da41c6bc494deaffbc77e2" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 38955 } } peers { permanent_uuid: "08873ffcfc304d1880333e420f583a1c" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 44551 } } peers { permanent_uuid: "f207febf699b41558ee44f228971f2d3" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 39195 } }
I20260925 22:09:05.117615 24207 raft_consensus.cc:374] T 9585b43dce534181b11e360d8cca8cc0 P 08873ffcfc304d1880333e420f583a1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:05.117660 24207 raft_consensus.cc:729] T 9585b43dce534181b11e360d8cca8cc0 P 08873ffcfc304d1880333e420f583a1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08873ffcfc304d1880333e420f583a1c, State: Initialized, Role: FOLLOWER
I20260925 22:09:05.117775 24207 consensus_queue.cc:260] T 9585b43dce534181b11e360d8cca8cc0 P 08873ffcfc304d1880333e420f583a1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef21fd7d25da41c6bc494deaffbc77e2" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 38955 } } peers { permanent_uuid: "08873ffcfc304d1880333e420f583a1c" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 44551 } } peers { permanent_uuid: "f207febf699b41558ee44f228971f2d3" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 39195 } }
I20260925 22:09:05.118011 24207 ts_tablet_manager.cc:1428] T 9585b43dce534181b11e360d8cca8cc0 P 08873ffcfc304d1880333e420f583a1c: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260925 22:09:05.118068 23927 heartbeater.cc:503] Master 127.5.0.126:34807 was elected leader, sending a full tablet report...
W20260925 22:09:05.200587 24059 tablet.cc:2369] T 9585b43dce534181b11e360d8cca8cc0 P f207febf699b41558ee44f228971f2d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:09:05.294648 24209 raft_consensus.cc:482] T 9585b43dce534181b11e360d8cca8cc0 P ef21fd7d25da41c6bc494deaffbc77e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:09:05.294770 24209 raft_consensus.cc:504] T 9585b43dce534181b11e360d8cca8cc0 P ef21fd7d25da41c6bc494deaffbc77e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef21fd7d25da41c6bc494deaffbc77e2" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 38955 } } peers { permanent_uuid: "08873ffcfc304d1880333e420f583a1c" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 44551 } } peers { permanent_uuid: "f207febf699b41558ee44f228971f2d3" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 39195 } }
I20260925 22:09:05.295069 24209 leader_election.cc:288] T 9585b43dce534181b11e360d8cca8cc0 P ef21fd7d25da41c6bc494deaffbc77e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08873ffcfc304d1880333e420f583a1c (127.5.0.65:44551), f207febf699b41558ee44f228971f2d3 (127.5.0.66:39195)
I20260925 22:09:05.298048 24191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:05.295294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:44551 (local address 127.5.0.67:50995)
0925 22:09:05.295462 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:05.295476 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:05.295537 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:05.295684 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:05.295685 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:05.295686 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:09:05.295762 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:05.295764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:05.296474 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:05.296475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:05.297226 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:05.297228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:05.297773 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:05.297774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:05.297848 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:05.297852 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:09:05.297926 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 22:09:05.297968 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142}
I20260925 22:09:05.298040 24202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:05.295474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:50995 (local address 127.5.0.65:44551)
0925 22:09:05.295552 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:05.295554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:05.295565 (+ 11us) server_negotiation.cc:407] Connection header received
0925 22:09:05.295596 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:05.295596 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:05.295614 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:05.295679 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:09:05.295834 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:05.296384 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:05.297328 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:05.297682 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:05.297771 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:05.297781 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:05.297934 (+ 153us) server_negotiation.cc:299] Negotiation successful
0925 22:09:05.297974 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260925 22:09:05.298247 24201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:05.295511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:50957 (local address 127.5.0.66:39195)
0925 22:09:05.295562 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:05.295564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:05.295717 (+ 153us) server_negotiation.cc:407] Connection header received
0925 22:09:05.295785 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:05.295786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:05.295796 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:05.295863 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:09:05.296064 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:05.296529 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:05.297479 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:05.297843 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:05.297927 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:05.297936 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:05.298104 (+ 168us) server_negotiation.cc:299] Negotiation successful
0925 22:09:05.298153 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 22:09:05.298413 24212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:05.295419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:39195 (local address 127.5.0.67:50957)
0925 22:09:05.295634 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:05.295642 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:05.295722 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:05.295865 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:05.295866 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:05.295868 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:09:05.295962 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:05.295964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:05.296620 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:05.296621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:05.297382 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:05.297384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:05.297928 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:05.297929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:05.298019 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:05.298022 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:09:05.298087 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 22:09:05.298261 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":143,"threads_started":1}
I20260925 22:09:05.298399 23881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9585b43dce534181b11e360d8cca8cc0" candidate_uuid: "ef21fd7d25da41c6bc494deaffbc77e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08873ffcfc304d1880333e420f583a1c" is_pre_election: true
I20260925 22:09:05.298621 23881 raft_consensus.cc:2454] T 9585b43dce534181b11e360d8cca8cc0 P 08873ffcfc304d1880333e420f583a1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef21fd7d25da41c6bc494deaffbc77e2 in term 0.
I20260925 22:09:05.298753 24012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9585b43dce534181b11e360d8cca8cc0" candidate_uuid: "ef21fd7d25da41c6bc494deaffbc77e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f207febf699b41558ee44f228971f2d3" is_pre_election: true
I20260925 22:09:05.298866 24078 leader_election.cc:302] T 9585b43dce534181b11e360d8cca8cc0 P ef21fd7d25da41c6bc494deaffbc77e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08873ffcfc304d1880333e420f583a1c, ef21fd7d25da41c6bc494deaffbc77e2; no voters:
I20260925 22:09:05.298910 24012 raft_consensus.cc:2454] T 9585b43dce534181b11e360d8cca8cc0 P f207febf699b41558ee44f228971f2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef21fd7d25da41c6bc494deaffbc77e2 in term 0.
I20260925 22:09:05.298986 24209 raft_consensus.cc:2781] T 9585b43dce534181b11e360d8cca8cc0 P ef21fd7d25da41c6bc494deaffbc77e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:09:05.299072 24209 raft_consensus.cc:482] T 9585b43dce534181b11e360d8cca8cc0 P ef21fd7d25da41c6bc494deaffbc77e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:09:05.299099 24209 raft_consensus.cc:3037] T 9585b43dce534181b11e360d8cca8cc0 P ef21fd7d25da41c6bc494deaffbc77e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:05.299795 24209 raft_consensus.cc:504] T 9585b43dce534181b11e360d8cca8cc0 P ef21fd7d25da41c6bc494deaffbc77e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef21fd7d25da41c6bc494deaffbc77e2" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 38955 } } peers { permanent_uuid: "08873ffcfc304d1880333e420f583a1c" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 44551 } } peers { permanent_uuid: "f207febf699b41558ee44f228971f2d3" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 39195 } }
I20260925 22:09:05.299969 24209 leader_election.cc:288] T 9585b43dce534181b11e360d8cca8cc0 P ef21fd7d25da41c6bc494deaffbc77e2 [CANDIDATE]: Term 1 election: Requested vote from peers 08873ffcfc304d1880333e420f583a1c (127.5.0.65:44551), f207febf699b41558ee44f228971f2d3 (127.5.0.66:39195)
I20260925 22:09:05.300124 23881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9585b43dce534181b11e360d8cca8cc0" candidate_uuid: "ef21fd7d25da41c6bc494deaffbc77e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08873ffcfc304d1880333e420f583a1c"
I20260925 22:09:05.300156 24012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9585b43dce534181b11e360d8cca8cc0" candidate_uuid: "ef21fd7d25da41c6bc494deaffbc77e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f207febf699b41558ee44f228971f2d3"
I20260925 22:09:05.300215 23881 raft_consensus.cc:3037] T 9585b43dce534181b11e360d8cca8cc0 P 08873ffcfc304d1880333e420f583a1c [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:05.300244 24012 raft_consensus.cc:3037] T 9585b43dce534181b11e360d8cca8cc0 P f207febf699b41558ee44f228971f2d3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:05.300935 24012 raft_consensus.cc:2454] T 9585b43dce534181b11e360d8cca8cc0 P f207febf699b41558ee44f228971f2d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef21fd7d25da41c6bc494deaffbc77e2 in term 1.
I20260925 22:09:05.300948 23881 raft_consensus.cc:2454] T 9585b43dce534181b11e360d8cca8cc0 P 08873ffcfc304d1880333e420f583a1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef21fd7d25da41c6bc494deaffbc77e2 in term 1.
I20260925 22:09:05.301103 24078 leader_election.cc:302] T 9585b43dce534181b11e360d8cca8cc0 P ef21fd7d25da41c6bc494deaffbc77e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef21fd7d25da41c6bc494deaffbc77e2, f207febf699b41558ee44f228971f2d3; no voters:
I20260925 22:09:05.301211 24209 raft_consensus.cc:2781] T 9585b43dce534181b11e360d8cca8cc0 P ef21fd7d25da41c6bc494deaffbc77e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:09:05.301353 24209 raft_consensus.cc:686] T 9585b43dce534181b11e360d8cca8cc0 P ef21fd7d25da41c6bc494deaffbc77e2 [term 1 LEADER]: Becoming Leader. State: Replica: ef21fd7d25da41c6bc494deaffbc77e2, State: Running, Role: LEADER
I20260925 22:09:05.301448 24209 consensus_queue.cc:237] T 9585b43dce534181b11e360d8cca8cc0 P ef21fd7d25da41c6bc494deaffbc77e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef21fd7d25da41c6bc494deaffbc77e2" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 38955 } } peers { permanent_uuid: "08873ffcfc304d1880333e420f583a1c" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 44551 } } peers { permanent_uuid: "f207febf699b41558ee44f228971f2d3" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 39195 } }
I20260925 22:09:05.302327 23738 catalog_manager.cc:5495] T 9585b43dce534181b11e360d8cca8cc0 P ef21fd7d25da41c6bc494deaffbc77e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef21fd7d25da41c6bc494deaffbc77e2 (127.5.0.67). New cstate: current_term: 1 leader_uuid: "ef21fd7d25da41c6bc494deaffbc77e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef21fd7d25da41c6bc494deaffbc77e2" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 38955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "08873ffcfc304d1880333e420f583a1c" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 44551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f207febf699b41558ee44f228971f2d3" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 39195 } health_report { overall_health: UNKNOWN } } }
W20260925 22:09:05.319811 23928 tablet.cc:2369] T 9585b43dce534181b11e360d8cca8cc0 P 08873ffcfc304d1880333e420f583a1c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 22:09:05.324488 24190 tablet.cc:2369] T 9585b43dce534181b11e360d8cca8cc0 P ef21fd7d25da41c6bc494deaffbc77e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:09:05.331059 24199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:05.328390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42470 (local address 127.5.0.67:38955)
0925 22:09:05.328430 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:05.328432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:05.328442 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:09:05.328548 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:05.328549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:05.328568 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:05.328632 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:09:05.328811 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:05.329455 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:05.330257 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:05.330484 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:05.330542 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:05.330733 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:09:05.330819 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:09:05.330899 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:05.330971 (+ 72us) server_negotiation.cc:299] Negotiation successful
0925 22:09:05.330979 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 22:09:05.333033 23881 raft_consensus.cc:1261] T 9585b43dce534181b11e360d8cca8cc0 P 08873ffcfc304d1880333e420f583a1c [term 1 FOLLOWER]: Refusing update from remote peer ef21fd7d25da41c6bc494deaffbc77e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:09:05.333025 24012 raft_consensus.cc:1261] T 9585b43dce534181b11e360d8cca8cc0 P f207febf699b41558ee44f228971f2d3 [term 1 FOLLOWER]: Refusing update from remote peer ef21fd7d25da41c6bc494deaffbc77e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:09:05.333384 24209 consensus_queue.cc:1035] T 9585b43dce534181b11e360d8cca8cc0 P ef21fd7d25da41c6bc494deaffbc77e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f207febf699b41558ee44f228971f2d3" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 39195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:09:05.333603 24213 consensus_queue.cc:1035] T 9585b43dce534181b11e360d8cca8cc0 P ef21fd7d25da41c6bc494deaffbc77e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08873ffcfc304d1880333e420f583a1c" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 44551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:09:05.338188 23738 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:09:05.338291 23738 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:09:05.339056 23738 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c38b71603c944e71a2f149b5b496e09e: Sending DeleteTablet for 3 replicas of tablet 9585b43dce534181b11e360d8cca8cc0
I20260925 22:09:05.339432 23861 tablet_service.cc:1514] Processing DeleteTablet for tablet 9585b43dce534181b11e360d8cca8cc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:09:05 UTC) from {username='slave'} at 127.0.0.1:33236
I20260925 22:09:05.339448 23992 tablet_service.cc:1514] Processing DeleteTablet for tablet 9585b43dce534181b11e360d8cca8cc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:09:05 UTC) from {username='slave'} at 127.0.0.1:38014
I20260925 22:09:05.339666 24123 tablet_service.cc:1514] Processing DeleteTablet for tablet 9585b43dce534181b11e360d8cca8cc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:09:05 UTC) from {username='slave'} at 127.0.0.1:42466
I20260925 22:09:05.339872 24227 tablet_replica.cc:330] T 9585b43dce534181b11e360d8cca8cc0 P 08873ffcfc304d1880333e420f583a1c: stopping tablet replica
I20260925 22:09:05.339999 24227 raft_consensus.cc:2229] T 9585b43dce534181b11e360d8cca8cc0 P 08873ffcfc304d1880333e420f583a1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:09:05.340106 24227 raft_consensus.cc:2258] T 9585b43dce534181b11e360d8cca8cc0 P 08873ffcfc304d1880333e420f583a1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:09:05.340253 24228 tablet_replica.cc:330] T 9585b43dce534181b11e360d8cca8cc0 P ef21fd7d25da41c6bc494deaffbc77e2: stopping tablet replica
I20260925 22:09:05.340345 24228 raft_consensus.cc:2229] T 9585b43dce534181b11e360d8cca8cc0 P ef21fd7d25da41c6bc494deaffbc77e2 [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:09:05.340430 24228 raft_consensus.cc:2258] T 9585b43dce534181b11e360d8cca8cc0 P ef21fd7d25da41c6bc494deaffbc77e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:09:05.340811 24227 ts_tablet_manager.cc:1905] T 9585b43dce534181b11e360d8cca8cc0 P 08873ffcfc304d1880333e420f583a1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:09:05.340935 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 23799
I20260925 22:09:05.341154 24228 ts_tablet_manager.cc:1905] T 9585b43dce534181b11e360d8cca8cc0 P ef21fd7d25da41c6bc494deaffbc77e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:09:05.342808 24228 ts_tablet_manager.cc:1918] T 9585b43dce534181b11e360d8cca8cc0 P ef21fd7d25da41c6bc494deaffbc77e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:09:05.342909 24228 log.cc:1196] T 9585b43dce534181b11e360d8cca8cc0 P ef21fd7d25da41c6bc494deaffbc77e2: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373908987421-5121-0/minicluster-data/ts-2/wal/wals/9585b43dce534181b11e360d8cca8cc0
I20260925 22:09:05.343158 24228 ts_tablet_manager.cc:1939] T 9585b43dce534181b11e360d8cca8cc0 P ef21fd7d25da41c6bc494deaffbc77e2: Deleting consensus metadata
I20260925 22:09:05.343534 23724 catalog_manager.cc:4841] TS ef21fd7d25da41c6bc494deaffbc77e2 (127.5.0.67:38955): tablet 9585b43dce534181b11e360d8cca8cc0 (table test-table [id=5198cfc0ad7c41e39a07daddbc7bc233]) successfully deleted
W20260925 22:09:05.347070 23725 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.0.65:44551 (error 108)
I20260925 22:09:05.347096 24226 tablet_replica.cc:330] T 9585b43dce534181b11e360d8cca8cc0 P f207febf699b41558ee44f228971f2d3: stopping tablet replica
I20260925 22:09:05.347223 24226 raft_consensus.cc:2229] T 9585b43dce534181b11e360d8cca8cc0 P f207febf699b41558ee44f228971f2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:09:05.347350 24226 raft_consensus.cc:2258] T 9585b43dce534181b11e360d8cca8cc0 P f207febf699b41558ee44f228971f2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:09:05.347434 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 23930
I20260925 22:09:05.348048 24200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:05.347530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:44551 (local address 127.0.0.1:33244)
0925 22:09:05.347966 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:05.348013 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.65:44551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413}
W20260925 22:09:05.348217 23725 catalog_manager.cc:4584] TS 08873ffcfc304d1880333e420f583a1c (127.5.0.65:44551): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9585b43dce534181b11e360d8cca8cc0: Network error: Client connection negotiation failed: client connection to 127.5.0.65:44551: connect: Connection refused (error 111)
I20260925 22:09:05.352478 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 24061
I20260925 22:09:05.352607 24200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:05.352476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:39195 (local address 127.0.0.1:38024)
0925 22:09:05.352539 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:05.352572 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.66:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260925 22:09:05.358011 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 23708
2026-09-25T22:09:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (845 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:09:05 dist-test-slave-7mqr krb5kdc[24232](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:09:05 dist-test-slave-7mqr krb5kdc[24232](info): set up 2 sockets
Sep 25 22:09:05 dist-test-slave-7mqr krb5kdc[24232](info): commencing operation
krb5kdc: starting...
W20260925 22:09:06.447676 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:09:06 dist-test-slave-7mqr krb5kdc[24232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:09:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:09:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.0.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
I20260925 22:09:06.560699 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:44077
--webserver_interface=127.5.0.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:34379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:44077
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:09:06.643707 24245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:06.644007 24245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:06.644038 24245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:06.645206 24245 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:09:06.645249 24245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:06.645263 24245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:06.645275 24245 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:09:06.645287 24245 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:09:06.646778 24245 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:34379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:44077
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:44077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:06.647042 24245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:06.647286 24245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:06.652024 24251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:06.652034 24253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:06.652034 24250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:06.652213 24245 server_base.cc:1034] running on GCE node
I20260925 22:09:06.652532 24245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:06.652925 24245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:06.654115 24245 hybrid_clock.cc:648] HybridClock initialized: now 1790374146654088 us; error 36 us; skew 500 ppm
Sep 25 22:09:06 dist-test-slave-7mqr krb5kdc[24232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374146, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:06.656756 24245 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:09:06.657241 24245 webserver.cc:460] Webserver started at http://127.5.0.126:44161/ using document root <none> and password file <none>
I20260925 22:09:06.657486 24245 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:06.657563 24245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:06.657709 24245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:06.658749 24245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "d5280868d2b84b1182dc2af8e93c12a6"
format_stamp: "Formatted at 2026-09-25 22:09:06 on dist-test-slave-7mqr"
server_key: "13601db97785b5932fd3ad5be9c9ecec"
server_key_iv: "0aa4da3c3cb2f911797b87a0688c11c4"
server_key_version: "encryptionkey@0"
I20260925 22:09:06.659112 24245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "d5280868d2b84b1182dc2af8e93c12a6"
format_stamp: "Formatted at 2026-09-25 22:09:06 on dist-test-slave-7mqr"
server_key: "13601db97785b5932fd3ad5be9c9ecec"
server_key_iv: "0aa4da3c3cb2f911797b87a0688c11c4"
server_key_version: "encryptionkey@0"
I20260925 22:09:06.660534 24245 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260925 22:09:06.661237 24260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:06.661435 24245 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:09:06.661559 24245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "d5280868d2b84b1182dc2af8e93c12a6"
format_stamp: "Formatted at 2026-09-25 22:09:06 on dist-test-slave-7mqr"
server_key: "13601db97785b5932fd3ad5be9c9ecec"
server_key_iv: "0aa4da3c3cb2f911797b87a0688c11c4"
server_key_version: "encryptionkey@0"
I20260925 22:09:06.661639 24245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:06.674625 24245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:06.676909 24245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:06.677099 24245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:06.681356 24245 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:44077
I20260925 22:09:06.681406 24312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:44077 every 8 connection(s)
I20260925 22:09:06.681957 24245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:09:06.682626 24313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:06.684886 24313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5280868d2b84b1182dc2af8e93c12a6: Bootstrap starting.
I20260925 22:09:06.685470 24313 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5280868d2b84b1182dc2af8e93c12a6: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:06.685770 24313 log.cc:824] T 00000000000000000000000000000000 P d5280868d2b84b1182dc2af8e93c12a6: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:06.686007 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 24245
I20260925 22:09:06.686144 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:09:06.686378 5121 external_mini_cluster.cc:1428] Setting key 394a37935daf9fb905f98771c3e3c6c6
I20260925 22:09:06.686667 24313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5280868d2b84b1182dc2af8e93c12a6: No bootstrap required, opened a new log
I20260925 22:09:06.688586 24313 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5280868d2b84b1182dc2af8e93c12a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5280868d2b84b1182dc2af8e93c12a6" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 44077 } }
I20260925 22:09:06.688786 24313 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5280868d2b84b1182dc2af8e93c12a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:06.688838 24313 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5280868d2b84b1182dc2af8e93c12a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5280868d2b84b1182dc2af8e93c12a6, State: Initialized, Role: FOLLOWER
I20260925 22:09:06.688964 24313 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5280868d2b84b1182dc2af8e93c12a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5280868d2b84b1182dc2af8e93c12a6" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 44077 } }
I20260925 22:09:06.689060 24313 raft_consensus.cc:388] T 00000000000000000000000000000000 P d5280868d2b84b1182dc2af8e93c12a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:09:06.689098 24313 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5280868d2b84b1182dc2af8e93c12a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:09:06.689152 24313 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5280868d2b84b1182dc2af8e93c12a6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:06.689845 24313 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5280868d2b84b1182dc2af8e93c12a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5280868d2b84b1182dc2af8e93c12a6" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 44077 } }
I20260925 22:09:06.689950 24313 leader_election.cc:302] T 00000000000000000000000000000000 P d5280868d2b84b1182dc2af8e93c12a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5280868d2b84b1182dc2af8e93c12a6; no voters:
I20260925 22:09:06.690111 24313 leader_election.cc:288] T 00000000000000000000000000000000 P d5280868d2b84b1182dc2af8e93c12a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:09:06.690164 24318 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5280868d2b84b1182dc2af8e93c12a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:09:06.690325 24318 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5280868d2b84b1182dc2af8e93c12a6 [term 1 LEADER]: Becoming Leader. State: Replica: d5280868d2b84b1182dc2af8e93c12a6, State: Running, Role: LEADER
I20260925 22:09:06.690346 24313 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5280868d2b84b1182dc2af8e93c12a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:09:06.690433 24318 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5280868d2b84b1182dc2af8e93c12a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5280868d2b84b1182dc2af8e93c12a6" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 44077 } }
I20260925 22:09:06.691529 24319 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5280868d2b84b1182dc2af8e93c12a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5280868d2b84b1182dc2af8e93c12a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5280868d2b84b1182dc2af8e93c12a6" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 44077 } } }
I20260925 22:09:06.691629 24319 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5280868d2b84b1182dc2af8e93c12a6 [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:06.691936 24320 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5280868d2b84b1182dc2af8e93c12a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5280868d2b84b1182dc2af8e93c12a6. Latest consensus state: current_term: 1 leader_uuid: "d5280868d2b84b1182dc2af8e93c12a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5280868d2b84b1182dc2af8e93c12a6" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 44077 } } }
I20260925 22:09:06.692006 24320 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5280868d2b84b1182dc2af8e93c12a6 [sys.catalog]: This master's current role is: LEADER
W20260925 22:09:06.692379 24333 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d5280868d2b84b1182dc2af8e93c12a6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:09:06.692443 24333 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:09:06.692523 24334 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:09:06.692682 24334 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:09:06.694231 24334 catalog_manager.cc:1347] Generated new cluster ID: 2b57caf637e449f4b0cdb03f429d6661
I20260925 22:09:06.694291 24334 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:09:06 dist-test-slave-7mqr krb5kdc[24232](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:09:06.699335 24317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:06.687140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45172 (local address 127.5.0.126:44077)
0925 22:09:06.687313 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:06.687321 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:06.687345 (+ 24us) server_negotiation.cc:407] Connection header received
0925 22:09:06.687884 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:06.687935 (+ 51us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:06.687998 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:06.688153 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:06.688856 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:06.690633 (+ 1777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:06.691732 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:06.692375 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:06.692489 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:06.695486 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:06.695490 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:06.695490 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:06.695502 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:06.697631 (+ 2129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:06.698031 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:06.698035 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:06.698035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:06.698145 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:06.698342 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:06.698343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:06.698343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:06.698740 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:06.698888 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:06.698986 (+ 98us) server_negotiation.cc:299] Negotiation successful
0925 22:09:06.699132 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":89,"threads_started":1}
I20260925 22:09:06.704594 24334 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:09:06.705142 24334 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:09:06.710638 24334 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5280868d2b84b1182dc2af8e93c12a6: Generated new TSK 0
I20260925 22:09:06.710829 24334 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.0.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
Entry for principal kudu/127.5.0.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab.
I20260925 22:09:06.738713 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.65:0
--local_ip_for_outbound_sockets=127.5.0.65
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:44077
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:34379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:09:06.819753 24339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:06.819995 24339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:06.820027 24339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:06.821211 24339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:06.821254 24339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:06.821281 24339 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.65
I20260925 22:09:06.822751 24339 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:34379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.65.keytab
--principal=kudu/127.5.0.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:44077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.65
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:06.823014 24339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:06.823210 24339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:06.828081 24347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:06.828086 24344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:06.828078 24345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:06.828179 24339 server_base.cc:1034] running on GCE node
I20260925 22:09:06.828470 24339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:06.828759 24339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:06.829931 24339 hybrid_clock.cc:648] HybridClock initialized: now 1790374146829922 us; error 35 us; skew 500 ppm
Sep 25 22:09:06 dist-test-slave-7mqr krb5kdc[24232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374146, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:06.832561 24339 init.cc:377] Logged in from keytab as kudu/127.5.0.65@KRBTEST.COM (short username kudu)
I20260925 22:09:06.832938 24339 webserver.cc:460] Webserver started at http://127.5.0.65:34759/ using document root <none> and password file <none>
I20260925 22:09:06.833205 24339 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:06.833250 24339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:06.833403 24339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:06.834445 24339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-0/data/instance:
uuid: "b8d633e95cf64095b71178c53d00b376"
format_stamp: "Formatted at 2026-09-25 22:09:06 on dist-test-slave-7mqr"
server_key: "431ed2a5da278314ee6a70dfd4e8be0c"
server_key_iv: "b6d8f6fa02dfbe637c53df10ce73e623"
server_key_version: "encryptionkey@0"
I20260925 22:09:06.834815 24339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance:
uuid: "b8d633e95cf64095b71178c53d00b376"
format_stamp: "Formatted at 2026-09-25 22:09:06 on dist-test-slave-7mqr"
server_key: "431ed2a5da278314ee6a70dfd4e8be0c"
server_key_iv: "b6d8f6fa02dfbe637c53df10ce73e623"
server_key_version: "encryptionkey@0"
I20260925 22:09:06.836205 24339 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 22:09:06.836953 24354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:06.837142 24339 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:09:06.837230 24339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-0/wal
uuid: "b8d633e95cf64095b71178c53d00b376"
format_stamp: "Formatted at 2026-09-25 22:09:06 on dist-test-slave-7mqr"
server_key: "431ed2a5da278314ee6a70dfd4e8be0c"
server_key_iv: "b6d8f6fa02dfbe637c53df10ce73e623"
server_key_version: "encryptionkey@0"
I20260925 22:09:06.837285 24339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:06.849622 24339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:06.851428 24339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:06.851533 24339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:06.851709 24339 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:09:06.852062 24339 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:09:06.852097 24339 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:06.852119 24339 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:09:06.852162 24339 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:06.857717 24339 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.65:34805
I20260925 22:09:06.857785 24467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.65:34805 every 8 connection(s)
I20260925 22:09:06.858108 24339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
I20260925 22:09:06.863514 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 24339
I20260925 22:09:06.863639 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance
I20260925 22:09:06.863859 5121 external_mini_cluster.cc:1428] Setting key 6934f88ff00da93ec4405af5fec29426
Sep 25 22:09:06 dist-test-slave-7mqr krb5kdc[24232](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374146, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.65@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:09:06.868115 24317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:06.858976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:54527 (local address 127.5.0.126:44077)
0925 22:09:06.859035 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:06.859037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:06.859271 (+ 234us) server_negotiation.cc:407] Connection header received
0925 22:09:06.859757 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:06.859758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:06.859766 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:06.859821 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:06.860777 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:06.861314 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:06.862317 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:06.862518 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:06.862574 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:06.864976 (+ 2402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:06.864977 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:06.864978 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:06.864985 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:06.866558 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:06.867058 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:06.867059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:06.867059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:06.867109 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:06.867386 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:06.867387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:06.867387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:06.867547 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:06.867618 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:06.867861 (+ 243us) server_negotiation.cc:299] Negotiation successful
0925 22:09:06.867976 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 22:09:06.868186 24470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:06.858958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:44077 (local address 127.5.0.65:54527)
0925 22:09:06.859137 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:06.859180 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:06.859672 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:06.859915 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:06.859919 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:06.860498 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:06.860688 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:06.860692 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:06.861412 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:06.861413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:06.862241 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:06.862243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:06.862613 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:06.862614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:06.862724 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:06.863274 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:09:06.863283 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:06.864885 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:06.866643 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:06.866649 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:06.866652 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:06.866980 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:06.867170 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:06.867171 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:06.867171 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:06.867304 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:06.867613 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:06.867617 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:06.867731 (+ 114us) client_negotiation.cc:769] Sending connection context
0925 22:09:06.867813 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 22:09:06.868000 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":101,"threads_started":1}
I20260925 22:09:06.868925 24468 heartbeater.cc:347] Connected to a master server at 127.5.0.126:44077
I20260925 22:09:06.869037 24468 heartbeater.cc:464] Registering TS with master...
I20260925 22:09:06.869220 24468 heartbeater.cc:511] Master 127.5.0.126:44077 requested a full tablet report, sending...
I20260925 22:09:06.869764 24277 ts_manager.cc:194] Registered new tserver with Master: b8d633e95cf64095b71178c53d00b376 (127.5.0.65:34805)
I20260925 22:09:06.870388 24277 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.65@KRBTEST.COM'} at 127.5.0.65:54527
WARNING: no policy specified for kudu/127.5.0.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
Entry for principal kudu/127.5.0.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab.
I20260925 22:09:06.890059 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.66:0
--local_ip_for_outbound_sockets=127.5.0.66
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:44077
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:34379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:09:06.970149 24473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:06.970376 24473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:06.970414 24473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:06.971556 24473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:06.971621 24473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:06.971669 24473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.66
I20260925 22:09:06.973176 24473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:34379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.66.keytab
--principal=kudu/127.5.0.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:44077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.66
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:06.973438 24473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:06.973663 24473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:06.978922 24478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:06.978899 24481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:06.978987 24479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:06.979084 24473 server_base.cc:1034] running on GCE node
I20260925 22:09:06.979301 24473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:06.979493 24473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:06.980628 24473 hybrid_clock.cc:648] HybridClock initialized: now 1790374146980593 us; error 52 us; skew 500 ppm
Sep 25 22:09:06 dist-test-slave-7mqr krb5kdc[24232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374146, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:06.983108 24473 init.cc:377] Logged in from keytab as kudu/127.5.0.66@KRBTEST.COM (short username kudu)
I20260925 22:09:06.983536 24473 webserver.cc:460] Webserver started at http://127.5.0.66:35677/ using document root <none> and password file <none>
I20260925 22:09:06.983767 24473 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:06.983812 24473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:06.983984 24473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:06.985039 24473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-1/data/instance:
uuid: "572ecb7a240c445ab322d7cb0aa0b71a"
format_stamp: "Formatted at 2026-09-25 22:09:06 on dist-test-slave-7mqr"
server_key: "a9fa5cd4daf10fd5e0fae6b9ee206e7b"
server_key_iv: "6a067f1ddce18a5455c7900656334932"
server_key_version: "encryptionkey@0"
I20260925 22:09:06.985396 24473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance:
uuid: "572ecb7a240c445ab322d7cb0aa0b71a"
format_stamp: "Formatted at 2026-09-25 22:09:06 on dist-test-slave-7mqr"
server_key: "a9fa5cd4daf10fd5e0fae6b9ee206e7b"
server_key_iv: "6a067f1ddce18a5455c7900656334932"
server_key_version: "encryptionkey@0"
I20260925 22:09:06.986781 24473 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:09:06.987519 24488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:06.987715 24473 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:09:06.987805 24473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-1/wal
uuid: "572ecb7a240c445ab322d7cb0aa0b71a"
format_stamp: "Formatted at 2026-09-25 22:09:06 on dist-test-slave-7mqr"
server_key: "a9fa5cd4daf10fd5e0fae6b9ee206e7b"
server_key_iv: "6a067f1ddce18a5455c7900656334932"
server_key_version: "encryptionkey@0"
I20260925 22:09:06.987862 24473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:07.014071 24473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:07.016562 24473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:07.016688 24473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:07.016891 24473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:09:07.017244 24473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:09:07.017280 24473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:07.017305 24473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:09:07.017359 24473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:07.023507 24473 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.66:36421
I20260925 22:09:07.023547 24601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.66:36421 every 8 connection(s)
I20260925 22:09:07.023924 24473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
I20260925 22:09:07.025166 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 24473
I20260925 22:09:07.025298 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance
I20260925 22:09:07.025554 5121 external_mini_cluster.cc:1428] Setting key 83d076fef0db25ffcad0cc93c40a4451
Sep 25 22:09:07 dist-test-slave-7mqr krb5kdc[24232](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374146, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.66@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.0.67@KRBTEST.COM; defaulting to no policy
I20260925 22:09:07.035141 24317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.024690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:37413 (local address 127.5.0.126:44077)
0925 22:09:07.024737 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:07.024740 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:07.024948 (+ 208us) server_negotiation.cc:407] Connection header received
0925 22:09:07.025500 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:07.025502 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:07.025517 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:07.025600 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:07.026749 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.027602 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.029137 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.029343 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.029404 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.031922 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:07.031923 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:07.031924 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:07.031932 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:07.033619 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:07.034061 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:07.034062 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:07.034062 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:07.034127 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:07.034408 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:07.034409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:07.034410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:07.034538 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:07.034625 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:07.034940 (+ 315us) server_negotiation.cc:299] Negotiation successful
0925 22:09:07.035035 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 22:09:07.035364 24604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.024649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:44077 (local address 127.5.0.66:37413)
0925 22:09:07.024811 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:07.024852 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:07.025374 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:07.025709 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:07.025724 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:07.026435 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:07.026656 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:07.026660 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.027736 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.027738 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:07.029060 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:07.029063 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.029423 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.029424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:07.029584 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.030136 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:09:07.030147 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:07.031792 (+ 1645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:07.033712 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:07.033719 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:07.033722 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:07.033968 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:07.034204 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:07.034204 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:07.034205 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:07.034322 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:07.034640 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:07.034646 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:07.034791 (+ 145us) client_negotiation.cc:769] Sending connection context
0925 22:09:07.034905 (+ 114us) client_negotiation.cc:240] Negotiation successful
0925 22:09:07.035096 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":88,"threads_started":1}
I20260925 22:09:07.035945 24602 heartbeater.cc:347] Connected to a master server at 127.5.0.126:44077
I20260925 22:09:07.036046 24602 heartbeater.cc:464] Registering TS with master...
I20260925 22:09:07.036198 24602 heartbeater.cc:511] Master 127.5.0.126:44077 requested a full tablet report, sending...
I20260925 22:09:07.036586 24277 ts_manager.cc:194] Registered new tserver with Master: 572ecb7a240c445ab322d7cb0aa0b71a (127.5.0.66:36421)
I20260925 22:09:07.037003 24277 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.66@KRBTEST.COM'} at 127.5.0.66:37413
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
Entry for principal kudu/127.5.0.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab.
I20260925 22:09:07.052748 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.67:0
--local_ip_for_outbound_sockets=127.5.0.67
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:44077
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:34379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:09:07.132193 24607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:07.132390 24607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:07.132412 24607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:07.133487 24607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:07.133518 24607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:07.133543 24607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.67
I20260925 22:09:07.134897 24607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:34379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.67.keytab
--principal=kudu/127.5.0.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:44077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.67
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:07.135073 24607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:07.135237 24607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:07.140036 24612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:07.140027 24615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:07.140106 24613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:07.140244 24607 server_base.cc:1034] running on GCE node
I20260925 22:09:07.140548 24607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:07.140798 24607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:07.141975 24607 hybrid_clock.cc:648] HybridClock initialized: now 1790374147141962 us; error 28 us; skew 500 ppm
Sep 25 22:09:07 dist-test-slave-7mqr krb5kdc[24232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374147, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:07.144579 24607 init.cc:377] Logged in from keytab as kudu/127.5.0.67@KRBTEST.COM (short username kudu)
I20260925 22:09:07.145031 24607 webserver.cc:460] Webserver started at http://127.5.0.67:40537/ using document root <none> and password file <none>
I20260925 22:09:07.145264 24607 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:07.145331 24607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:07.145484 24607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:07.146490 24607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-2/data/instance:
uuid: "4da471597c0e492fb5433cc37cd8aefa"
format_stamp: "Formatted at 2026-09-25 22:09:07 on dist-test-slave-7mqr"
server_key: "4b1a3b5cd548b4379571f265f5e350e3"
server_key_iv: "9b15515731667fd33d2e7e8fed3e50f8"
server_key_version: "encryptionkey@0"
I20260925 22:09:07.146852 24607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance:
uuid: "4da471597c0e492fb5433cc37cd8aefa"
format_stamp: "Formatted at 2026-09-25 22:09:07 on dist-test-slave-7mqr"
server_key: "4b1a3b5cd548b4379571f265f5e350e3"
server_key_iv: "9b15515731667fd33d2e7e8fed3e50f8"
server_key_version: "encryptionkey@0"
I20260925 22:09:07.148339 24607 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:09:07.149286 24622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:07.149454 24607 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:09:07.149524 24607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-2/wal
uuid: "4da471597c0e492fb5433cc37cd8aefa"
format_stamp: "Formatted at 2026-09-25 22:09:07 on dist-test-slave-7mqr"
server_key: "4b1a3b5cd548b4379571f265f5e350e3"
server_key_iv: "9b15515731667fd33d2e7e8fed3e50f8"
server_key_version: "encryptionkey@0"
I20260925 22:09:07.149574 24607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:07.165915 24607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:07.168306 24607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:07.168426 24607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:07.168602 24607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:09:07.168838 24607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:09:07.168866 24607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:07.168889 24607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:09:07.168905 24607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:07.174678 24607 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.67:36611
I20260925 22:09:07.174754 24735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.67:36611 every 8 connection(s)
I20260925 22:09:07.175071 24607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
I20260925 22:09:07.177253 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 24607
I20260925 22:09:07.177374 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance
I20260925 22:09:07.177594 5121 external_mini_cluster.cc:1428] Setting key 61301176ff629e1dbf5bd84fdfc97ac9
Sep 25 22:09:07 dist-test-slave-7mqr krb5kdc[24232](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374147, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.67@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:09:07.184336 24317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.175756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:53789 (local address 127.5.0.126:44077)
0925 22:09:07.175837 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:07.175839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:07.176074 (+ 235us) server_negotiation.cc:407] Connection header received
0925 22:09:07.176534 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:07.176535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:07.176544 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:07.176595 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:07.177516 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.178319 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.179335 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.179523 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.179577 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.181700 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:07.181702 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:07.181702 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:07.181710 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:07.182925 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:07.183357 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:07.183358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:07.183359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:07.183418 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:07.183671 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:07.183672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:07.183672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:07.183801 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:07.183875 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:07.184174 (+ 299us) server_negotiation.cc:299] Negotiation successful
0925 22:09:07.184247 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 22:09:07.184535 24738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.175720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:44077 (local address 127.5.0.67:53789)
0925 22:09:07.175936 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:07.175981 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:07.176454 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:07.176645 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:07.176648 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:07.177188 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:07.177421 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:07.177427 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.178407 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.178408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:07.179265 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:07.179268 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.179596 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.179596 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:07.179689 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.180232 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:09:07.180247 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:07.181602 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:07.183047 (+ 1445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:07.183053 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:07.183055 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:07.183272 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:07.183497 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:07.183497 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:07.183498 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:07.183597 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:07.183866 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:07.183872 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:07.184037 (+ 165us) client_negotiation.cc:769] Sending connection context
0925 22:09:07.184141 (+ 104us) client_negotiation.cc:240] Negotiation successful
0925 22:09:07.184292 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":100,"threads_started":1}
I20260925 22:09:07.185058 24736 heartbeater.cc:347] Connected to a master server at 127.5.0.126:44077
I20260925 22:09:07.185132 24736 heartbeater.cc:464] Registering TS with master...
I20260925 22:09:07.185318 24736 heartbeater.cc:511] Master 127.5.0.126:44077 requested a full tablet report, sending...
I20260925 22:09:07.185724 24277 ts_manager.cc:194] Registered new tserver with Master: 4da471597c0e492fb5433cc37cd8aefa (127.5.0.67:36611)
I20260925 22:09:07.186107 24277 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.0.67@KRBTEST.COM'} at 127.5.0.67:53789
I20260925 22:09:07.189213 5121 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:09:07.196502 24317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.190610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45176 (local address 127.5.0.126:44077)
0925 22:09:07.190681 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:07.190683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:07.190794 (+ 111us) server_negotiation.cc:407] Connection header received
0925 22:09:07.190866 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:07.190866 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:07.190876 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:07.190919 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:07.191607 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.192254 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.193036 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.193239 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.193313 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.194407 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:07.194408 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:07.194408 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:07.194415 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:07.195500 (+ 1085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:07.195836 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:07.195837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:07.195837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:07.195912 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:07.196114 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:07.196115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:07.196115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:07.196230 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:07.196304 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:07.196389 (+ 85us) server_negotiation.cc:299] Negotiation successful
0925 22:09:07.196441 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 22:09:07.197924 24277 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45176:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:09:07.199473 24277 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:09:07.205539 24744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.201557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:36421 (local address 127.0.0.1:34258)
0925 22:09:07.201787 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:07.201822 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:07.201904 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:07.202122 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:07.202124 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:07.202127 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:09:07.202188 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:07.202190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.203144 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.203145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:07.204480 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:07.204483 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.205234 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.205235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:07.205325 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.205338 (+ 13us) client_negotiation.cc:769] Sending connection context
0925 22:09:07.205404 (+ 66us) client_negotiation.cc:240] Negotiation successful
0925 22:09:07.205454 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":51,"threads_started":1}
I20260925 22:09:07.205642 24745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.201618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:34805 (local address 127.0.0.1:54520)
0925 22:09:07.202171 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:07.202179 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:07.202251 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:07.202526 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:07.202527 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:07.202529 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:09:07.202560 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:07.202562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.203758 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.203758 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:07.204531 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:07.204533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.205382 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.205383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:07.205437 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.205440 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:09:07.205505 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 22:09:07.205537 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":38,"threads_started":1}
I20260925 22:09:07.205848 24746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.201592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:36611 (local address 127.0.0.1:38398)
0925 22:09:07.202023 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:07.202033 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:07.202092 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:07.202287 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:07.202288 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:07.202290 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:09:07.202343 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:07.202344 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.203667 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.203668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:07.204755 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:07.204757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.205620 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.205621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:07.205694 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.205698 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:09:07.205726 (+ 28us) client_negotiation.cc:240] Negotiation successful
0925 22:09:07.205760 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"mutex_wait_us":32,"thread_start_us":50,"threads_started":1}
I20260925 22:09:07.205863 24748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.201791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34258 (local address 127.5.0.66:36421)
0925 22:09:07.201961 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:07.201966 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:07.201976 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:09:07.202012 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:07.202014 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:07.202030 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:07.202118 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:09:07.202273 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.203040 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.204559 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.205152 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.205236 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.205294 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:07.205736 (+ 442us) server_negotiation.cc:299] Negotiation successful
0925 22:09:07.205785 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":78,"threads_started":1}
I20260925 22:09:07.206074 24749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.201799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54520 (local address 127.5.0.65:34805)
0925 22:09:07.201977 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:07.201981 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:07.202226 (+ 245us) server_negotiation.cc:407] Connection header received
0925 22:09:07.202339 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:07.202341 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:07.202356 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:07.202409 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:09:07.202638 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.203654 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.204620 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.205297 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.205381 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.205435 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:07.205521 (+ 86us) server_negotiation.cc:299] Negotiation successful
0925 22:09:07.205569 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":77,"threads_started":1}
I20260925 22:09:07.206247 24747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.201660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38398 (local address 127.5.0.67:36611)
0925 22:09:07.201988 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:07.201992 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:07.202077 (+ 85us) server_negotiation.cc:407] Connection header received
0925 22:09:07.202167 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:07.202169 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:07.202186 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:07.202417 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:09:07.202480 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.203557 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.204834 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.205526 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.206028 (+ 502us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.206094 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:07.206121 (+ 27us) server_negotiation.cc:299] Negotiation successful
0925 22:09:07.206174 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":74,"threads_started":1}
I20260925 22:09:07.206564 24536 tablet_service.cc:1467] Processing CreateTablet for tablet 89b29ad7987641e8b3449779836d1b5a (DEFAULT_TABLE table=test-table [id=7cc645b0d7824fa0abd261e1887b6223]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:09:07.206839 24536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89b29ad7987641e8b3449779836d1b5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:07.206853 24402 tablet_service.cc:1467] Processing CreateTablet for tablet 89b29ad7987641e8b3449779836d1b5a (DEFAULT_TABLE table=test-table [id=7cc645b0d7824fa0abd261e1887b6223]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:09:07.206853 24670 tablet_service.cc:1467] Processing CreateTablet for tablet 89b29ad7987641e8b3449779836d1b5a (DEFAULT_TABLE table=test-table [id=7cc645b0d7824fa0abd261e1887b6223]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:09:07.207165 24402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89b29ad7987641e8b3449779836d1b5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:07.207165 24670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89b29ad7987641e8b3449779836d1b5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:07.209321 24750 tablet_bootstrap.cc:492] T 89b29ad7987641e8b3449779836d1b5a P 572ecb7a240c445ab322d7cb0aa0b71a: Bootstrap starting.
I20260925 22:09:07.210018 24750 tablet_bootstrap.cc:654] T 89b29ad7987641e8b3449779836d1b5a P 572ecb7a240c445ab322d7cb0aa0b71a: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:07.210335 24752 tablet_bootstrap.cc:492] T 89b29ad7987641e8b3449779836d1b5a P 4da471597c0e492fb5433cc37cd8aefa: Bootstrap starting.
I20260925 22:09:07.210840 24750 log.cc:824] T 89b29ad7987641e8b3449779836d1b5a P 572ecb7a240c445ab322d7cb0aa0b71a: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:07.211028 24752 tablet_bootstrap.cc:654] T 89b29ad7987641e8b3449779836d1b5a P 4da471597c0e492fb5433cc37cd8aefa: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:07.211210 24751 tablet_bootstrap.cc:492] T 89b29ad7987641e8b3449779836d1b5a P b8d633e95cf64095b71178c53d00b376: Bootstrap starting.
I20260925 22:09:07.211570 24752 log.cc:824] T 89b29ad7987641e8b3449779836d1b5a P 4da471597c0e492fb5433cc37cd8aefa: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:07.211853 24751 tablet_bootstrap.cc:654] T 89b29ad7987641e8b3449779836d1b5a P b8d633e95cf64095b71178c53d00b376: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:07.212286 24752 tablet_bootstrap.cc:492] T 89b29ad7987641e8b3449779836d1b5a P 4da471597c0e492fb5433cc37cd8aefa: No bootstrap required, opened a new log
I20260925 22:09:07.212374 24752 ts_tablet_manager.cc:1397] T 89b29ad7987641e8b3449779836d1b5a P 4da471597c0e492fb5433cc37cd8aefa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:09:07.212530 24751 log.cc:824] T 89b29ad7987641e8b3449779836d1b5a P b8d633e95cf64095b71178c53d00b376: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:07.212424 24750 tablet_bootstrap.cc:492] T 89b29ad7987641e8b3449779836d1b5a P 572ecb7a240c445ab322d7cb0aa0b71a: No bootstrap required, opened a new log
I20260925 22:09:07.212658 24750 ts_tablet_manager.cc:1397] T 89b29ad7987641e8b3449779836d1b5a P 572ecb7a240c445ab322d7cb0aa0b71a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 22:09:07.213268 24751 tablet_bootstrap.cc:492] T 89b29ad7987641e8b3449779836d1b5a P b8d633e95cf64095b71178c53d00b376: No bootstrap required, opened a new log
I20260925 22:09:07.213338 24751 ts_tablet_manager.cc:1397] T 89b29ad7987641e8b3449779836d1b5a P b8d633e95cf64095b71178c53d00b376: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 22:09:07.213994 24752 raft_consensus.cc:348] T 89b29ad7987641e8b3449779836d1b5a P 4da471597c0e492fb5433cc37cd8aefa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572ecb7a240c445ab322d7cb0aa0b71a" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 36421 } } peers { permanent_uuid: "4da471597c0e492fb5433cc37cd8aefa" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 36611 } } peers { permanent_uuid: "b8d633e95cf64095b71178c53d00b376" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 34805 } }
I20260925 22:09:07.214150 24752 raft_consensus.cc:374] T 89b29ad7987641e8b3449779836d1b5a P 4da471597c0e492fb5433cc37cd8aefa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:07.214192 24752 raft_consensus.cc:729] T 89b29ad7987641e8b3449779836d1b5a P 4da471597c0e492fb5433cc37cd8aefa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4da471597c0e492fb5433cc37cd8aefa, State: Initialized, Role: FOLLOWER
I20260925 22:09:07.214303 24752 consensus_queue.cc:260] T 89b29ad7987641e8b3449779836d1b5a P 4da471597c0e492fb5433cc37cd8aefa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572ecb7a240c445ab322d7cb0aa0b71a" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 36421 } } peers { permanent_uuid: "4da471597c0e492fb5433cc37cd8aefa" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 36611 } } peers { permanent_uuid: "b8d633e95cf64095b71178c53d00b376" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 34805 } }
I20260925 22:09:07.214565 24752 ts_tablet_manager.cc:1428] T 89b29ad7987641e8b3449779836d1b5a P 4da471597c0e492fb5433cc37cd8aefa: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:09:07.214546 24750 raft_consensus.cc:348] T 89b29ad7987641e8b3449779836d1b5a P 572ecb7a240c445ab322d7cb0aa0b71a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572ecb7a240c445ab322d7cb0aa0b71a" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 36421 } } peers { permanent_uuid: "4da471597c0e492fb5433cc37cd8aefa" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 36611 } } peers { permanent_uuid: "b8d633e95cf64095b71178c53d00b376" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 34805 } }
I20260925 22:09:07.214716 24736 heartbeater.cc:503] Master 127.5.0.126:44077 was elected leader, sending a full tablet report...
I20260925 22:09:07.214744 24750 raft_consensus.cc:374] T 89b29ad7987641e8b3449779836d1b5a P 572ecb7a240c445ab322d7cb0aa0b71a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:07.214784 24750 raft_consensus.cc:729] T 89b29ad7987641e8b3449779836d1b5a P 572ecb7a240c445ab322d7cb0aa0b71a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 572ecb7a240c445ab322d7cb0aa0b71a, State: Initialized, Role: FOLLOWER
I20260925 22:09:07.214888 24750 consensus_queue.cc:260] T 89b29ad7987641e8b3449779836d1b5a P 572ecb7a240c445ab322d7cb0aa0b71a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572ecb7a240c445ab322d7cb0aa0b71a" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 36421 } } peers { permanent_uuid: "4da471597c0e492fb5433cc37cd8aefa" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 36611 } } peers { permanent_uuid: "b8d633e95cf64095b71178c53d00b376" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 34805 } }
I20260925 22:09:07.215158 24750 ts_tablet_manager.cc:1428] T 89b29ad7987641e8b3449779836d1b5a P 572ecb7a240c445ab322d7cb0aa0b71a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:09:07.215353 24602 heartbeater.cc:503] Master 127.5.0.126:44077 was elected leader, sending a full tablet report...
I20260925 22:09:07.215281 24751 raft_consensus.cc:348] T 89b29ad7987641e8b3449779836d1b5a P b8d633e95cf64095b71178c53d00b376 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572ecb7a240c445ab322d7cb0aa0b71a" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 36421 } } peers { permanent_uuid: "4da471597c0e492fb5433cc37cd8aefa" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 36611 } } peers { permanent_uuid: "b8d633e95cf64095b71178c53d00b376" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 34805 } }
I20260925 22:09:07.215433 24751 raft_consensus.cc:374] T 89b29ad7987641e8b3449779836d1b5a P b8d633e95cf64095b71178c53d00b376 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:07.215466 24751 raft_consensus.cc:729] T 89b29ad7987641e8b3449779836d1b5a P b8d633e95cf64095b71178c53d00b376 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8d633e95cf64095b71178c53d00b376, State: Initialized, Role: FOLLOWER
I20260925 22:09:07.215571 24751 consensus_queue.cc:260] T 89b29ad7987641e8b3449779836d1b5a P b8d633e95cf64095b71178c53d00b376 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572ecb7a240c445ab322d7cb0aa0b71a" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 36421 } } peers { permanent_uuid: "4da471597c0e492fb5433cc37cd8aefa" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 36611 } } peers { permanent_uuid: "b8d633e95cf64095b71178c53d00b376" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 34805 } }
I20260925 22:09:07.215840 24751 ts_tablet_manager.cc:1428] T 89b29ad7987641e8b3449779836d1b5a P b8d633e95cf64095b71178c53d00b376: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260925 22:09:07.215862 24468 heartbeater.cc:503] Master 127.5.0.126:44077 was elected leader, sending a full tablet report...
I20260925 22:09:07.259264 24757 raft_consensus.cc:482] T 89b29ad7987641e8b3449779836d1b5a P 572ecb7a240c445ab322d7cb0aa0b71a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:09:07.259380 24757 raft_consensus.cc:504] T 89b29ad7987641e8b3449779836d1b5a P 572ecb7a240c445ab322d7cb0aa0b71a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572ecb7a240c445ab322d7cb0aa0b71a" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 36421 } } peers { permanent_uuid: "4da471597c0e492fb5433cc37cd8aefa" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 36611 } } peers { permanent_uuid: "b8d633e95cf64095b71178c53d00b376" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 34805 } }
I20260925 22:09:07.259732 24757 leader_election.cc:288] T 89b29ad7987641e8b3449779836d1b5a P 572ecb7a240c445ab322d7cb0aa0b71a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4da471597c0e492fb5433cc37cd8aefa (127.5.0.67:36611), b8d633e95cf64095b71178c53d00b376 (127.5.0.65:34805)
I20260925 22:09:07.263193 24747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.259953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:47513 (local address 127.5.0.67:36611)
0925 22:09:07.260010 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:07.260012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:07.260024 (+ 12us) server_negotiation.cc:407] Connection header received
0925 22:09:07.260090 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:07.260091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:07.260109 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:07.260161 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:09:07.260341 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.261138 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.262331 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.262724 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.262778 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.262787 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:07.263069 (+ 282us) server_negotiation.cc:299] Negotiation successful
0925 22:09:07.263117 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 22:09:07.263234 24749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.259999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:51175 (local address 127.5.0.65:34805)
0925 22:09:07.260052 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:07.260054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:07.260240 (+ 186us) server_negotiation.cc:407] Connection header received
0925 22:09:07.260266 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:07.260267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:07.260277 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:07.260347 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:09:07.260464 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.261146 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.262382 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.262876 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.262948 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.262958 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:07.263123 (+ 165us) server_negotiation.cc:299] Negotiation successful
0925 22:09:07.263172 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260925 22:09:07.263229 24604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.259901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:36611 (local address 127.5.0.66:47513)
0925 22:09:07.259961 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:07.259973 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:07.260027 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:07.260185 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:07.260186 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:07.260187 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:09:07.260270 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:07.260272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.261234 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.261235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:07.262243 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:07.262245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.262899 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.262900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:07.262987 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.262991 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:09:07.263046 (+ 55us) client_negotiation.cc:240] Negotiation successful
0925 22:09:07.263100 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":4}
I20260925 22:09:07.263237 24759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.259923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:34805 (local address 127.5.0.66:51175)
0925 22:09:07.260124 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:07.260132 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:07.260208 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:07.260363 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:07.260363 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:07.260364 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:09:07.260405 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:07.260406 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.261234 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.261235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:07.262255 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:07.262259 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.262956 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.262957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:07.263016 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.263038 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 22:09:07.263097 (+ 59us) client_negotiation.cc:240] Negotiation successful
0925 22:09:07.263135 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":69,"thread_start_us":128,"threads_started":1}
I20260925 22:09:07.263616 24690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89b29ad7987641e8b3449779836d1b5a" candidate_uuid: "572ecb7a240c445ab322d7cb0aa0b71a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4da471597c0e492fb5433cc37cd8aefa" is_pre_election: true
I20260925 22:09:07.263744 24422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89b29ad7987641e8b3449779836d1b5a" candidate_uuid: "572ecb7a240c445ab322d7cb0aa0b71a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d633e95cf64095b71178c53d00b376" is_pre_election: true
I20260925 22:09:07.263789 24690 raft_consensus.cc:2454] T 89b29ad7987641e8b3449779836d1b5a P 4da471597c0e492fb5433cc37cd8aefa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 572ecb7a240c445ab322d7cb0aa0b71a in term 0.
I20260925 22:09:07.263873 24422 raft_consensus.cc:2454] T 89b29ad7987641e8b3449779836d1b5a P b8d633e95cf64095b71178c53d00b376 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 572ecb7a240c445ab322d7cb0aa0b71a in term 0.
I20260925 22:09:07.264063 24490 leader_election.cc:302] T 89b29ad7987641e8b3449779836d1b5a P 572ecb7a240c445ab322d7cb0aa0b71a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4da471597c0e492fb5433cc37cd8aefa, 572ecb7a240c445ab322d7cb0aa0b71a; no voters:
I20260925 22:09:07.264215 24757 raft_consensus.cc:2781] T 89b29ad7987641e8b3449779836d1b5a P 572ecb7a240c445ab322d7cb0aa0b71a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:09:07.264273 24757 raft_consensus.cc:482] T 89b29ad7987641e8b3449779836d1b5a P 572ecb7a240c445ab322d7cb0aa0b71a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:09:07.264302 24757 raft_consensus.cc:3037] T 89b29ad7987641e8b3449779836d1b5a P 572ecb7a240c445ab322d7cb0aa0b71a [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:07.265138 24757 raft_consensus.cc:504] T 89b29ad7987641e8b3449779836d1b5a P 572ecb7a240c445ab322d7cb0aa0b71a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572ecb7a240c445ab322d7cb0aa0b71a" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 36421 } } peers { permanent_uuid: "4da471597c0e492fb5433cc37cd8aefa" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 36611 } } peers { permanent_uuid: "b8d633e95cf64095b71178c53d00b376" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 34805 } }
I20260925 22:09:07.265278 24757 leader_election.cc:288] T 89b29ad7987641e8b3449779836d1b5a P 572ecb7a240c445ab322d7cb0aa0b71a [CANDIDATE]: Term 1 election: Requested vote from peers 4da471597c0e492fb5433cc37cd8aefa (127.5.0.67:36611), b8d633e95cf64095b71178c53d00b376 (127.5.0.65:34805)
I20260925 22:09:07.265440 24690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89b29ad7987641e8b3449779836d1b5a" candidate_uuid: "572ecb7a240c445ab322d7cb0aa0b71a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4da471597c0e492fb5433cc37cd8aefa"
I20260925 22:09:07.265457 24422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89b29ad7987641e8b3449779836d1b5a" candidate_uuid: "572ecb7a240c445ab322d7cb0aa0b71a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d633e95cf64095b71178c53d00b376"
I20260925 22:09:07.265548 24690 raft_consensus.cc:3037] T 89b29ad7987641e8b3449779836d1b5a P 4da471597c0e492fb5433cc37cd8aefa [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:07.265564 24422 raft_consensus.cc:3037] T 89b29ad7987641e8b3449779836d1b5a P b8d633e95cf64095b71178c53d00b376 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:07.266232 24422 raft_consensus.cc:2454] T 89b29ad7987641e8b3449779836d1b5a P b8d633e95cf64095b71178c53d00b376 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 572ecb7a240c445ab322d7cb0aa0b71a in term 1.
I20260925 22:09:07.266284 24690 raft_consensus.cc:2454] T 89b29ad7987641e8b3449779836d1b5a P 4da471597c0e492fb5433cc37cd8aefa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 572ecb7a240c445ab322d7cb0aa0b71a in term 1.
I20260925 22:09:07.266444 24492 leader_election.cc:302] T 89b29ad7987641e8b3449779836d1b5a P 572ecb7a240c445ab322d7cb0aa0b71a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 572ecb7a240c445ab322d7cb0aa0b71a, b8d633e95cf64095b71178c53d00b376; no voters:
I20260925 22:09:07.266533 24757 raft_consensus.cc:2781] T 89b29ad7987641e8b3449779836d1b5a P 572ecb7a240c445ab322d7cb0aa0b71a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:09:07.266700 24757 raft_consensus.cc:686] T 89b29ad7987641e8b3449779836d1b5a P 572ecb7a240c445ab322d7cb0aa0b71a [term 1 LEADER]: Becoming Leader. State: Replica: 572ecb7a240c445ab322d7cb0aa0b71a, State: Running, Role: LEADER
I20260925 22:09:07.266783 24757 consensus_queue.cc:237] T 89b29ad7987641e8b3449779836d1b5a P 572ecb7a240c445ab322d7cb0aa0b71a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572ecb7a240c445ab322d7cb0aa0b71a" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 36421 } } peers { permanent_uuid: "4da471597c0e492fb5433cc37cd8aefa" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 36611 } } peers { permanent_uuid: "b8d633e95cf64095b71178c53d00b376" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 34805 } }
I20260925 22:09:07.267539 24276 catalog_manager.cc:5495] T 89b29ad7987641e8b3449779836d1b5a P 572ecb7a240c445ab322d7cb0aa0b71a reported cstate change: term changed from 0 to 1, leader changed from <none> to 572ecb7a240c445ab322d7cb0aa0b71a (127.5.0.66). New cstate: current_term: 1 leader_uuid: "572ecb7a240c445ab322d7cb0aa0b71a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572ecb7a240c445ab322d7cb0aa0b71a" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4da471597c0e492fb5433cc37cd8aefa" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 36611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8d633e95cf64095b71178c53d00b376" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 34805 } health_report { overall_health: UNKNOWN } } }
W20260925 22:09:07.274423 24603 tablet.cc:2369] T 89b29ad7987641e8b3449779836d1b5a P 572ecb7a240c445ab322d7cb0aa0b71a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:09:07.280463 24748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.278250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34262 (local address 127.5.0.66:36421)
0925 22:09:07.278299 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:07.278301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:07.278310 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:09:07.278381 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:07.278382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:07.278397 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:07.278450 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:09:07.278581 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.279031 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.279755 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.280016 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.280068 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.280214 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:09:07.280302 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:09:07.280360 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:07.280398 (+ 38us) server_negotiation.cc:299] Negotiation successful
0925 22:09:07.280413 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260925 22:09:07.282089 24422 raft_consensus.cc:1261] T 89b29ad7987641e8b3449779836d1b5a P b8d633e95cf64095b71178c53d00b376 [term 1 FOLLOWER]: Refusing update from remote peer 572ecb7a240c445ab322d7cb0aa0b71a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:09:07.282097 24690 raft_consensus.cc:1261] T 89b29ad7987641e8b3449779836d1b5a P 4da471597c0e492fb5433cc37cd8aefa [term 1 FOLLOWER]: Refusing update from remote peer 572ecb7a240c445ab322d7cb0aa0b71a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:09:07.282445 24760 consensus_queue.cc:1035] T 89b29ad7987641e8b3449779836d1b5a P 572ecb7a240c445ab322d7cb0aa0b71a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8d633e95cf64095b71178c53d00b376" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 34805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:09:07.282569 24757 consensus_queue.cc:1035] T 89b29ad7987641e8b3449779836d1b5a P 572ecb7a240c445ab322d7cb0aa0b71a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4da471597c0e492fb5433cc37cd8aefa" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 36611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:09:07.284798 24763 mvcc.cc:205] Tried to move back new op lower bound from 7333372507264847872 to 7333372507205177344. Current Snapshot: MvccSnapshot[applied={T|T < 7333372507264847872}]
I20260925 22:09:07.285084 24764 mvcc.cc:205] Tried to move back new op lower bound from 7333372507264847872 to 7333372507205177344. Current Snapshot: MvccSnapshot[applied={T|T < 7333372507264847872}]
I20260925 22:09:07.285878 24765 mvcc.cc:205] Tried to move back new op lower bound from 7333372507264847872 to 7333372507205177344. Current Snapshot: MvccSnapshot[applied={T|T < 7333372507264847872}]
I20260925 22:09:07.287134 24276 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45176:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:09:07.287227 24276 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45176:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:09:07.287999 24276 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d5280868d2b84b1182dc2af8e93c12a6: Sending DeleteTablet for 3 replicas of tablet 89b29ad7987641e8b3449779836d1b5a
I20260925 22:09:07.288311 24536 tablet_service.cc:1514] Processing DeleteTablet for tablet 89b29ad7987641e8b3449779836d1b5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:09:07 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:34258
I20260925 22:09:07.288374 24402 tablet_service.cc:1514] Processing DeleteTablet for tablet 89b29ad7987641e8b3449779836d1b5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:09:07 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:54520
I20260925 22:09:07.288393 24670 tablet_service.cc:1514] Processing DeleteTablet for tablet 89b29ad7987641e8b3449779836d1b5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:09:07 UTC) from {username='kudu', principal='kudu/127.5.0.126@KRBTEST.COM'} at 127.0.0.1:38398
I20260925 22:09:07.288563 24773 tablet_replica.cc:330] T 89b29ad7987641e8b3449779836d1b5a P 572ecb7a240c445ab322d7cb0aa0b71a: stopping tablet replica
I20260925 22:09:07.288655 24775 tablet_replica.cc:330] T 89b29ad7987641e8b3449779836d1b5a P 4da471597c0e492fb5433cc37cd8aefa: stopping tablet replica
I20260925 22:09:07.288661 24773 raft_consensus.cc:2229] T 89b29ad7987641e8b3449779836d1b5a P 572ecb7a240c445ab322d7cb0aa0b71a [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:09:07.288741 24775 raft_consensus.cc:2229] T 89b29ad7987641e8b3449779836d1b5a P 4da471597c0e492fb5433cc37cd8aefa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:09:07.288750 24773 raft_consensus.cc:2258] T 89b29ad7987641e8b3449779836d1b5a P 572ecb7a240c445ab322d7cb0aa0b71a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:09:07.288838 24775 raft_consensus.cc:2258] T 89b29ad7987641e8b3449779836d1b5a P 4da471597c0e492fb5433cc37cd8aefa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:09:07.288890 24774 tablet_replica.cc:330] T 89b29ad7987641e8b3449779836d1b5a P b8d633e95cf64095b71178c53d00b376: stopping tablet replica
I20260925 22:09:07.288993 24774 raft_consensus.cc:2229] T 89b29ad7987641e8b3449779836d1b5a P b8d633e95cf64095b71178c53d00b376 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:09:07.289105 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 24339
I20260925 22:09:07.289175 24774 raft_consensus.cc:2258] T 89b29ad7987641e8b3449779836d1b5a P b8d633e95cf64095b71178c53d00b376 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:09:07.289361 24773 ts_tablet_manager.cc:1905] T 89b29ad7987641e8b3449779836d1b5a P 572ecb7a240c445ab322d7cb0aa0b71a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:09:07.290987 24773 ts_tablet_manager.cc:1918] T 89b29ad7987641e8b3449779836d1b5a P 572ecb7a240c445ab322d7cb0aa0b71a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:09:07.291074 24773 log.cc:1196] T 89b29ad7987641e8b3449779836d1b5a P 572ecb7a240c445ab322d7cb0aa0b71a: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-1/wal/wals/89b29ad7987641e8b3449779836d1b5a
I20260925 22:09:07.291384 24773 ts_tablet_manager.cc:1939] T 89b29ad7987641e8b3449779836d1b5a P 572ecb7a240c445ab322d7cb0aa0b71a: Deleting consensus metadata
I20260925 22:09:07.291760 24261 catalog_manager.cc:4841] TS 572ecb7a240c445ab322d7cb0aa0b71a (127.5.0.66:36421): tablet 89b29ad7987641e8b3449779836d1b5a (table test-table [id=7cc645b0d7824fa0abd261e1887b6223]) successfully deleted
W20260925 22:09:07.295365 24264 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.0.65:34805 (error 108)
I20260925 22:09:07.295606 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 24473
I20260925 22:09:07.296370 24775 ts_tablet_manager.cc:1905] T 89b29ad7987641e8b3449779836d1b5a P 4da471597c0e492fb5433cc37cd8aefa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:09:07.296494 24744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.295748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:34805 (local address 127.0.0.1:54536)
0925 22:09:07.296427 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:07.296466 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.65:34805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":657}
W20260925 22:09:07.296627 24264 catalog_manager.cc:4584] TS b8d633e95cf64095b71178c53d00b376 (127.5.0.65:34805): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 89b29ad7987641e8b3449779836d1b5a: Network error: Client connection negotiation failed: client connection to 127.5.0.65:34805: connect: Connection refused (error 111)
I20260925 22:09:07.297869 24775 ts_tablet_manager.cc:1918] T 89b29ad7987641e8b3449779836d1b5a P 4da471597c0e492fb5433cc37cd8aefa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:09:07.297921 24775 log.cc:1196] T 89b29ad7987641e8b3449779836d1b5a P 4da471597c0e492fb5433cc37cd8aefa: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373908987421-5121-0/minicluster-data/ts-2/wal/wals/89b29ad7987641e8b3449779836d1b5a
I20260925 22:09:07.298158 24775 ts_tablet_manager.cc:1939] T 89b29ad7987641e8b3449779836d1b5a P 4da471597c0e492fb5433cc37cd8aefa: Deleting consensus metadata
I20260925 22:09:07.298542 24262 catalog_manager.cc:4841] TS 4da471597c0e492fb5433cc37cd8aefa (127.5.0.67:36611): tablet 89b29ad7987641e8b3449779836d1b5a (table test-table [id=7cc645b0d7824fa0abd261e1887b6223]) successfully deleted
I20260925 22:09:07.301641 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 24607
I20260925 22:09:07.306128 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 24245
2026-09-25T22:09:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1948 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-25T22:09:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:09:07Z Disabled control of system clock
I20260925 22:09:07.333370 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:41681
--webserver_interface=127.5.0.126
--webserver_port=0
--builtin_ntp_servers=127.5.0.84:33237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:41681
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 22:09:07.412622 24780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:07.412829 24780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:07.412849 24780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:07.413995 24780 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:09:07.414037 24780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:07.414050 24780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:07.414062 24780 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:09:07.414113 24780 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:09:07.415411 24780 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:33237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:41681
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:41681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:07.415664 24780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:07.415979 24780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:07.420651 24785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:07.420707 24786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:07.420809 24780 server_base.cc:1034] running on GCE node
W20260925 22:09:07.420830 24788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:07.421079 24780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:07.421363 24780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:07.422580 24780 hybrid_clock.cc:648] HybridClock initialized: now 1790374147422555 us; error 35 us; skew 500 ppm
I20260925 22:09:07.423859 24780 webserver.cc:460] Webserver started at http://127.5.0.126:36641/ using document root <none> and password file <none>
I20260925 22:09:07.424162 24780 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:07.424211 24780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:07.424350 24780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:07.425501 24780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "db55b0f4c00741ac98b69f235b0e831f"
format_stamp: "Formatted at 2026-09-25 22:09:07 on dist-test-slave-7mqr"
server_key: "99e9a7861c181e4fe20fa4e6527a1f23"
server_key_iv: "db9f28085b639f94ad32411f70bcc72b"
server_key_version: "encryptionkey@0"
I20260925 22:09:07.425859 24780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "db55b0f4c00741ac98b69f235b0e831f"
format_stamp: "Formatted at 2026-09-25 22:09:07 on dist-test-slave-7mqr"
server_key: "99e9a7861c181e4fe20fa4e6527a1f23"
server_key_iv: "db9f28085b639f94ad32411f70bcc72b"
server_key_version: "encryptionkey@0"
I20260925 22:09:07.427156 24780 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:09:07.427875 24794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:07.428076 24780 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:09:07.428162 24780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "db55b0f4c00741ac98b69f235b0e831f"
format_stamp: "Formatted at 2026-09-25 22:09:07 on dist-test-slave-7mqr"
server_key: "99e9a7861c181e4fe20fa4e6527a1f23"
server_key_iv: "db9f28085b639f94ad32411f70bcc72b"
server_key_version: "encryptionkey@0"
I20260925 22:09:07.428216 24780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:07.434819 24780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:07.435070 24780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:07.435173 24780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:07.438854 24780 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:41681
I20260925 22:09:07.438884 24846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:41681 every 8 connection(s)
I20260925 22:09:07.439240 24780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:09:07.440209 24847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:07.442389 24847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db55b0f4c00741ac98b69f235b0e831f: Bootstrap starting.
I20260925 22:09:07.442970 24847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db55b0f4c00741ac98b69f235b0e831f: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:07.443270 24847 log.cc:824] T 00000000000000000000000000000000 P db55b0f4c00741ac98b69f235b0e831f: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:07.443982 24847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db55b0f4c00741ac98b69f235b0e831f: No bootstrap required, opened a new log
I20260925 22:09:07.445536 24847 raft_consensus.cc:348] T 00000000000000000000000000000000 P db55b0f4c00741ac98b69f235b0e831f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db55b0f4c00741ac98b69f235b0e831f" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41681 } }
I20260925 22:09:07.445672 24847 raft_consensus.cc:374] T 00000000000000000000000000000000 P db55b0f4c00741ac98b69f235b0e831f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:07.445693 24847 raft_consensus.cc:729] T 00000000000000000000000000000000 P db55b0f4c00741ac98b69f235b0e831f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db55b0f4c00741ac98b69f235b0e831f, State: Initialized, Role: FOLLOWER
I20260925 22:09:07.445852 24847 consensus_queue.cc:260] T 00000000000000000000000000000000 P db55b0f4c00741ac98b69f235b0e831f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db55b0f4c00741ac98b69f235b0e831f" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41681 } }
I20260925 22:09:07.445916 24847 raft_consensus.cc:388] T 00000000000000000000000000000000 P db55b0f4c00741ac98b69f235b0e831f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:09:07.445937 24847 raft_consensus.cc:482] T 00000000000000000000000000000000 P db55b0f4c00741ac98b69f235b0e831f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:09:07.446005 24847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db55b0f4c00741ac98b69f235b0e831f [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:07.446527 24847 raft_consensus.cc:504] T 00000000000000000000000000000000 P db55b0f4c00741ac98b69f235b0e831f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db55b0f4c00741ac98b69f235b0e831f" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41681 } }
I20260925 22:09:07.446637 24847 leader_election.cc:302] T 00000000000000000000000000000000 P db55b0f4c00741ac98b69f235b0e831f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db55b0f4c00741ac98b69f235b0e831f; no voters:
I20260925 22:09:07.446815 24847 leader_election.cc:288] T 00000000000000000000000000000000 P db55b0f4c00741ac98b69f235b0e831f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:09:07.446862 24850 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db55b0f4c00741ac98b69f235b0e831f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:09:07.447075 24850 raft_consensus.cc:686] T 00000000000000000000000000000000 P db55b0f4c00741ac98b69f235b0e831f [term 1 LEADER]: Becoming Leader. State: Replica: db55b0f4c00741ac98b69f235b0e831f, State: Running, Role: LEADER
I20260925 22:09:07.447084 24847 sys_catalog.cc:564] T 00000000000000000000000000000000 P db55b0f4c00741ac98b69f235b0e831f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:09:07.447188 24850 consensus_queue.cc:237] T 00000000000000000000000000000000 P db55b0f4c00741ac98b69f235b0e831f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db55b0f4c00741ac98b69f235b0e831f" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41681 } }
I20260925 22:09:07.447443 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 24780
I20260925 22:09:07.447553 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:09:07.447751 5121 external_mini_cluster.cc:1428] Setting key b3c38dac36323465c8258ecc78503509
I20260925 22:09:07.448032 24852 sys_catalog.cc:455] T 00000000000000000000000000000000 P db55b0f4c00741ac98b69f235b0e831f [sys.catalog]: SysCatalogTable state changed. Reason: New leader db55b0f4c00741ac98b69f235b0e831f. Latest consensus state: current_term: 1 leader_uuid: "db55b0f4c00741ac98b69f235b0e831f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db55b0f4c00741ac98b69f235b0e831f" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41681 } } }
I20260925 22:09:07.448115 24852 sys_catalog.cc:458] T 00000000000000000000000000000000 P db55b0f4c00741ac98b69f235b0e831f [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:07.448212 24851 sys_catalog.cc:455] T 00000000000000000000000000000000 P db55b0f4c00741ac98b69f235b0e831f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db55b0f4c00741ac98b69f235b0e831f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db55b0f4c00741ac98b69f235b0e831f" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 41681 } } }
I20260925 22:09:07.448297 24851 sys_catalog.cc:458] T 00000000000000000000000000000000 P db55b0f4c00741ac98b69f235b0e831f [sys.catalog]: This master's current role is: LEADER
W20260925 22:09:07.449033 24867 catalog_manager.cc:1558] T 00000000000000000000000000000000 P db55b0f4c00741ac98b69f235b0e831f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:09:07.449095 24867 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:09:07.449142 24859 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:09:07.449293 24859 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:09:07.453437 24866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.448410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50056 (local address 127.5.0.126:41681)
0925 22:09:07.448847 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:07.448855 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:07.448888 (+ 33us) server_negotiation.cc:407] Connection header received
0925 22:09:07.448997 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:07.449002 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:07.449096 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:07.449246 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:07.449472 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.450448 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.451528 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.452005 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.452100 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.452761 (+ 661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:07.452764 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:07.452765 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:07.452926 (+ 161us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:07.452961 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:07.452971 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:07.453027 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:07.453119 (+ 92us) server_negotiation.cc:299] Negotiation successful
0925 22:09:07.453178 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":75,"threads_started":1}
I20260925 22:09:07.454078 24859 catalog_manager.cc:1347] Generated new cluster ID: 77b247491dfe4bcd8fd501a4a2d1083f
I20260925 22:09:07.454133 24859 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:09:07.478909 24859 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:09:07.479526 24859 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:09:07.481834 24859 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db55b0f4c00741ac98b69f235b0e831f: Generated new TSK 0
I20260925 22:09:07.482016 24859 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:09:07.491292 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.65:0
--local_ip_for_outbound_sockets=127.5.0.65
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:41681
--builtin_ntp_servers=127.5.0.84:33237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 22:09:07.571290 24871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:07.571492 24871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:07.571513 24871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:07.572700 24871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:07.572741 24871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:07.572764 24871 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.65
I20260925 22:09:07.574080 24871 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:33237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.0.65
--webserver_port=0
--tserver_master_addrs=127.5.0.126:41681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.65
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:07.574319 24871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:07.574615 24871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:07.579517 24879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:07.579493 24876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:07.579502 24877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:07.579725 24871 server_base.cc:1034] running on GCE node
I20260925 22:09:07.579990 24871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:07.580248 24871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:07.581410 24871 hybrid_clock.cc:648] HybridClock initialized: now 1790374147581402 us; error 37 us; skew 500 ppm
I20260925 22:09:07.582698 24871 webserver.cc:460] Webserver started at http://127.5.0.65:35685/ using document root <none> and password file <none>
I20260925 22:09:07.582952 24871 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:07.583029 24871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:07.583182 24871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:07.584300 24871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-0/data/instance:
uuid: "2af9c96d863f44b2b9d87dc118d7fb97"
format_stamp: "Formatted at 2026-09-25 22:09:07 on dist-test-slave-7mqr"
server_key: "bfa0f95ed4a1a1e3b16963e787ae1521"
server_key_iv: "43a13a45c7c4be2ff8d0869ca1f41243"
server_key_version: "encryptionkey@0"
I20260925 22:09:07.584657 24871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance:
uuid: "2af9c96d863f44b2b9d87dc118d7fb97"
format_stamp: "Formatted at 2026-09-25 22:09:07 on dist-test-slave-7mqr"
server_key: "bfa0f95ed4a1a1e3b16963e787ae1521"
server_key_iv: "43a13a45c7c4be2ff8d0869ca1f41243"
server_key_version: "encryptionkey@0"
I20260925 22:09:07.585951 24871 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 22:09:07.586653 24885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:07.586823 24871 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:09:07.586921 24871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-0/wal
uuid: "2af9c96d863f44b2b9d87dc118d7fb97"
format_stamp: "Formatted at 2026-09-25 22:09:07 on dist-test-slave-7mqr"
server_key: "bfa0f95ed4a1a1e3b16963e787ae1521"
server_key_iv: "43a13a45c7c4be2ff8d0869ca1f41243"
server_key_version: "encryptionkey@0"
I20260925 22:09:07.587009 24871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:07.606902 24871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:07.607213 24871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:07.607365 24871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:07.607594 24871 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:09:07.607980 24871 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:09:07.608038 24871 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:07.608079 24871 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:09:07.608106 24871 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:07.613852 24871 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.65:43653
I20260925 22:09:07.613903 24998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.65:43653 every 8 connection(s)
I20260925 22:09:07.614223 24871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-0/data/info.pb
I20260925 22:09:07.616096 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 24871
I20260925 22:09:07.616209 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-0/wal/instance
I20260925 22:09:07.616402 5121 external_mini_cluster.cc:1428] Setting key 958ad374fe8b8bc99b4349cdad843f0b
I20260925 22:09:07.617584 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.66:0
--local_ip_for_outbound_sockets=127.5.0.66
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:41681
--builtin_ntp_servers=127.5.0.84:33237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 22:09:07.619923 24866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.614948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.65:53347 (local address 127.5.0.126:41681)
0925 22:09:07.615021 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:07.615023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:07.615230 (+ 207us) server_negotiation.cc:407] Connection header received
0925 22:09:07.615691 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:07.615692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:07.615703 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:07.615761 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:07.616550 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.617182 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.618273 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.618607 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.618682 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.619628 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:07.619629 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:07.619630 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:07.619635 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:07.619648 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:07.619652 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:07.619697 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:07.619808 (+ 111us) server_negotiation.cc:299] Negotiation successful
0925 22:09:07.619831 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 22:09:07.620193 25001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.614930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41681 (local address 127.5.0.65:53347)
0925 22:09:07.615105 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:07.615145 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:07.615611 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:07.615813 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:07.615816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:07.615823 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:07.616443 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:07.616449 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.617273 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.617274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:07.618179 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:07.618182 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.618715 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.618717 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:07.618887 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.619464 (+ 577us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:09:07.619465 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:07.619504 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:09:07.619535 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:07.619723 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:07.619727 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:07.619730 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:09:07.619798 (+ 68us) client_negotiation.cc:240] Negotiation successful
0925 22:09:07.619930 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":80,"threads_started":1}
I20260925 22:09:07.620712 24999 heartbeater.cc:347] Connected to a master server at 127.5.0.126:41681
I20260925 22:09:07.620808 24999 heartbeater.cc:464] Registering TS with master...
I20260925 22:09:07.621017 24999 heartbeater.cc:511] Master 127.5.0.126:41681 requested a full tablet report, sending...
I20260925 22:09:07.621556 24811 ts_manager.cc:194] Registered new tserver with Master: 2af9c96d863f44b2b9d87dc118d7fb97 (127.5.0.65:43653)
I20260925 22:09:07.622159 24811 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.0.65:53347
W20260925 22:09:07.698969 25002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:07.699190 25002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:07.699213 25002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:07.700441 25002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:07.700484 25002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:07.700515 25002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.66
I20260925 22:09:07.701893 25002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:33237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.0.66
--webserver_port=0
--tserver_master_addrs=127.5.0.126:41681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.66
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:07.702157 25002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:07.702459 25002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:07.707352 25010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:07.707360 25008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:07.707343 25007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:07.707667 25002 server_base.cc:1034] running on GCE node
I20260925 22:09:07.707931 25002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:07.708184 25002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:07.709332 25002 hybrid_clock.cc:648] HybridClock initialized: now 1790374147709315 us; error 31 us; skew 500 ppm
I20260925 22:09:07.710503 25002 webserver.cc:460] Webserver started at http://127.5.0.66:35657/ using document root <none> and password file <none>
I20260925 22:09:07.710763 25002 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:07.710834 25002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:07.710988 25002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:07.712127 25002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-1/data/instance:
uuid: "ad6948765ea142c58c3b13702d835ca2"
format_stamp: "Formatted at 2026-09-25 22:09:07 on dist-test-slave-7mqr"
server_key: "64950582e1e388175c0640c4b3318e0b"
server_key_iv: "f420c1cd9d8e21b1a5718466b4c022e0"
server_key_version: "encryptionkey@0"
I20260925 22:09:07.712527 25002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance:
uuid: "ad6948765ea142c58c3b13702d835ca2"
format_stamp: "Formatted at 2026-09-25 22:09:07 on dist-test-slave-7mqr"
server_key: "64950582e1e388175c0640c4b3318e0b"
server_key_iv: "f420c1cd9d8e21b1a5718466b4c022e0"
server_key_version: "encryptionkey@0"
I20260925 22:09:07.713824 25002 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:09:07.714524 25016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:07.714700 25002 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:09:07.714797 25002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-1/wal
uuid: "ad6948765ea142c58c3b13702d835ca2"
format_stamp: "Formatted at 2026-09-25 22:09:07 on dist-test-slave-7mqr"
server_key: "64950582e1e388175c0640c4b3318e0b"
server_key_iv: "f420c1cd9d8e21b1a5718466b4c022e0"
server_key_version: "encryptionkey@0"
I20260925 22:09:07.714884 25002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:07.731007 25002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:07.731343 25002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:07.731506 25002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:07.731761 25002 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:09:07.732141 25002 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:09:07.732199 25002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:07.732242 25002 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:09:07.732268 25002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:07.737969 25002 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.66:39279
I20260925 22:09:07.738047 25129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.66:39279 every 8 connection(s)
I20260925 22:09:07.738389 25002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-1/data/info.pb
I20260925 22:09:07.742993 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 25002
I20260925 22:09:07.743150 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-1/wal/instance
I20260925 22:09:07.743389 5121 external_mini_cluster.cc:1428] Setting key 4ebf2fa8cbc9a23d762c6aee991ba421
I20260925 22:09:07.743587 24866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.739129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.66:33143 (local address 127.5.0.126:41681)
0925 22:09:07.739201 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:07.739204 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:07.739470 (+ 266us) server_negotiation.cc:407] Connection header received
0925 22:09:07.740015 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:07.740016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:07.740025 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:07.740079 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:07.740733 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.741222 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.742171 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.742347 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.742396 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.743298 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:07.743299 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:07.743300 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:07.743306 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:07.743325 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:07.743330 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:07.743382 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:07.743474 (+ 92us) server_negotiation.cc:299] Negotiation successful
0925 22:09:07.743505 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 22:09:07.743815 25132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.739132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41681 (local address 127.5.0.66:33143)
0925 22:09:07.739342 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:07.739375 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:07.739862 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:07.740137 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:07.740140 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:07.740149 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:07.740662 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:07.740666 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.741298 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.741298 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:07.742085 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:07.742087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.742417 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.742418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:07.742536 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.743096 (+ 560us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:09:07.743097 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:07.743154 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:09:07.743188 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:07.743389 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:07.743393 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:07.743395 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 22:09:07.743473 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 22:09:07.743587 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":96,"threads_started":1}
I20260925 22:09:07.744274 25130 heartbeater.cc:347] Connected to a master server at 127.5.0.126:41681
I20260925 22:09:07.744367 25130 heartbeater.cc:464] Registering TS with master...
I20260925 22:09:07.744511 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.0.67:0
--local_ip_for_outbound_sockets=127.5.0.67
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:41681
--builtin_ntp_servers=127.5.0.84:33237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 22:09:07.744562 25130 heartbeater.cc:511] Master 127.5.0.126:41681 requested a full tablet report, sending...
I20260925 22:09:07.744916 24811 ts_manager.cc:194] Registered new tserver with Master: ad6948765ea142c58c3b13702d835ca2 (127.5.0.66:39279)
I20260925 22:09:07.745252 24811 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.0.66:33143
W20260925 22:09:07.824690 25133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:07.824908 25133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:07.824944 25133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:07.826128 25133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:07.826196 25133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:07.826237 25133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.0.67
I20260925 22:09:07.827618 25133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:33237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.0.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.0.67
--webserver_port=0
--tserver_master_addrs=127.5.0.126:41681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.0.67
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:07.827926 25133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:07.828215 25133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:07.833164 25139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:07.833195 25133 server_base.cc:1034] running on GCE node
W20260925 22:09:07.833177 25141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:07.833177 25138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:07.833608 25133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:07.833847 25133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:07.835006 25133 hybrid_clock.cc:648] HybridClock initialized: now 1790374147835002 us; error 37 us; skew 500 ppm
I20260925 22:09:07.836321 25133 webserver.cc:460] Webserver started at http://127.5.0.67:45545/ using document root <none> and password file <none>
I20260925 22:09:07.836567 25133 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:07.836639 25133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:07.836774 25133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:07.837819 25133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-2/data/instance:
uuid: "5aab8263b1b44d34ac9ce7e079ee7077"
format_stamp: "Formatted at 2026-09-25 22:09:07 on dist-test-slave-7mqr"
server_key: "c005b0a2ffce928b3528bde9edcdbd4d"
server_key_iv: "b7cb5a049fafd43b809a85fb799f5df7"
server_key_version: "encryptionkey@0"
I20260925 22:09:07.838184 25133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance:
uuid: "5aab8263b1b44d34ac9ce7e079ee7077"
format_stamp: "Formatted at 2026-09-25 22:09:07 on dist-test-slave-7mqr"
server_key: "c005b0a2ffce928b3528bde9edcdbd4d"
server_key_iv: "b7cb5a049fafd43b809a85fb799f5df7"
server_key_version: "encryptionkey@0"
I20260925 22:09:07.839460 25133 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 22:09:07.840240 25147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:07.840402 25133 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:07.840499 25133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-2/wal
uuid: "5aab8263b1b44d34ac9ce7e079ee7077"
format_stamp: "Formatted at 2026-09-25 22:09:07 on dist-test-slave-7mqr"
server_key: "c005b0a2ffce928b3528bde9edcdbd4d"
server_key_iv: "b7cb5a049fafd43b809a85fb799f5df7"
server_key_version: "encryptionkey@0"
I20260925 22:09:07.840590 25133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:07.856956 25133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:07.857296 25133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:07.857450 25133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:07.857684 25133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:09:07.858037 25133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:09:07.858093 25133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:07.858134 25133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:09:07.858160 25133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:07.863824 25133 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.67:42213
I20260925 22:09:07.863873 25260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.67:42213 every 8 connection(s)
I20260925 22:09:07.864248 25133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-2/data/info.pb
I20260925 22:09:07.869447 24866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.864928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:47447 (local address 127.5.0.126:41681)
0925 22:09:07.864973 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:07.864975 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:07.865213 (+ 238us) server_negotiation.cc:407] Connection header received
0925 22:09:07.865758 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:07.865759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:07.865772 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:07.865823 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:07.866528 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.867033 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.868076 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.868284 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.868358 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.869158 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:07.869160 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:07.869160 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:07.869165 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:07.869180 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:07.869184 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:07.869237 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:07.869336 (+ 99us) server_negotiation.cc:299] Negotiation successful
0925 22:09:07.869366 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:09:07.869481 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 25133
I20260925 22:09:07.869632 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-2/wal/instance
I20260925 22:09:07.869684 25263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.864910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:41681 (local address 127.5.0.67:47447)
0925 22:09:07.865089 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:07.865123 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:07.865640 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:07.865891 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:07.865895 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:07.865904 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:07.866421 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:07.866425 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.867130 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.867131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:07.867984 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:07.867986 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.868378 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.868379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:07.868516 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.869019 (+ 503us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:09:07.869019 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:07.869047 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:09:07.869068 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:07.869246 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:07.869250 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:07.869253 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:09:07.869338 (+ 85us) client_negotiation.cc:240] Negotiation successful
0925 22:09:07.869435 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":78,"threads_started":1}
I20260925 22:09:07.869841 5121 external_mini_cluster.cc:1428] Setting key ea2f9a88d5e4b8a11f0297c3c7e79767
I20260925 22:09:07.870105 25261 heartbeater.cc:347] Connected to a master server at 127.5.0.126:41681
I20260925 22:09:07.870198 25261 heartbeater.cc:464] Registering TS with master...
I20260925 22:09:07.870373 25261 heartbeater.cc:511] Master 127.5.0.126:41681 requested a full tablet report, sending...
I20260925 22:09:07.870710 24811 ts_manager.cc:194] Registered new tserver with Master: 5aab8263b1b44d34ac9ce7e079ee7077 (127.5.0.67:42213)
I20260925 22:09:07.871083 24811 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.0.67:47447
I20260925 22:09:07.881438 5121 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:09:07.886075 24866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.883175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50070 (local address 127.5.0.126:41681)
0925 22:09:07.883238 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:07.883240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:07.883358 (+ 118us) server_negotiation.cc:407] Connection header received
0925 22:09:07.883411 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:07.883411 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:07.883421 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:07.883462 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:07.883635 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.884173 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.884990 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.885164 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.885214 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.885871 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:07.885872 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:07.885872 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:07.885876 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:07.885890 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:07.885894 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:07.885937 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:07.885990 (+ 53us) server_negotiation.cc:299] Negotiation successful
0925 22:09:07.886014 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 22:09:07.887607 24811 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50070:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:09:07.889178 24811 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:09:07.895381 25271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.891469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59074 (local address 127.5.0.66:39279)
0925 22:09:07.892149 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:07.892154 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:07.892163 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:09:07.892198 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:07.892200 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:07.892213 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:07.892285 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:09:07.892386 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.893071 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.894474 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.894956 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.895066 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.895163 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:07.895256 (+ 93us) server_negotiation.cc:299] Negotiation successful
0925 22:09:07.895292 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":101,"threads_started":1}
I20260925 22:09:07.895370 25269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.891378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:39279 (local address 127.0.0.1:59074)
0925 22:09:07.891491 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:07.891564 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:07.891631 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:07.892283 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:07.892284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:07.892287 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:09:07.892325 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:07.892327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.893180 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.893181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:07.894382 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:07.894385 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.895072 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.895073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:07.895153 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.895166 (+ 13us) client_negotiation.cc:769] Sending connection context
0925 22:09:07.895241 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 22:09:07.895293 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":1,"thread_start_us":85,"threads_started":1}
I20260925 22:09:07.895540 25273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.891628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35934 (local address 127.5.0.67:42213)
0925 22:09:07.891788 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:07.891793 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:07.891803 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:09:07.891841 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:07.891843 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:07.891860 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:07.891959 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:09:07.892210 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.892990 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.894272 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.895017 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.895085 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.895163 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:07.895411 (+ 248us) server_negotiation.cc:299] Negotiation successful
0925 22:09:07.895452 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":95,"threads_started":1}
I20260925 22:09:07.895618 25270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.891382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.67:42213 (local address 127.0.0.1:35934)
0925 22:09:07.891706 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:07.891716 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:07.891765 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:07.892051 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:07.892055 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:07.892057 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:09:07.892123 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:07.892125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.893076 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.893077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:07.894182 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:07.894185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.895195 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.895196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:07.895254 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.895258 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:09:07.895301 (+ 43us) client_negotiation.cc:240] Negotiation successful
0925 22:09:07.895350 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"mutex_wait_us":40,"thread_start_us":63,"threads_started":1}
I20260925 22:09:07.896090 25064 tablet_service.cc:1467] Processing CreateTablet for tablet 26f6f6a2b7104f1897367eabf9eca68f (DEFAULT_TABLE table=test-table [id=03bae0ddaff14f6a81cc3840f4b7d643]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:09:07.896471 25064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26f6f6a2b7104f1897367eabf9eca68f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:07.896356 25195 tablet_service.cc:1467] Processing CreateTablet for tablet 26f6f6a2b7104f1897367eabf9eca68f (DEFAULT_TABLE table=test-table [id=03bae0ddaff14f6a81cc3840f4b7d643]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:09:07.896704 25195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26f6f6a2b7104f1897367eabf9eca68f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:07.897716 25272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.891600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:43653 (local address 127.0.0.1:60688)
0925 22:09:07.891947 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:07.891958 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:07.892016 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:07.892258 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:07.892259 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:07.892261 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:09:07.892323 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:07.892326 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.894417 (+ 2091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.894418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:07.896463 (+ 2045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:07.896465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.897494 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.897496 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:07.897558 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.897562 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:09:07.897614 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 22:09:07.897653 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":54,"threads_started":1}
I20260925 22:09:07.898005 25274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.891734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60688 (local address 127.5.0.65:43653)
0925 22:09:07.891929 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:07.891933 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:07.892023 (+ 90us) server_negotiation.cc:407] Connection header received
0925 22:09:07.892080 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:07.892083 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:07.892098 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:07.892183 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:09:07.893198 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.894332 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.896569 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.897298 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.897370 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.897447 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:07.897885 (+ 438us) server_negotiation.cc:299] Negotiation successful
0925 22:09:07.897927 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":88,"threads_started":1}
I20260925 22:09:07.898622 24933 tablet_service.cc:1467] Processing CreateTablet for tablet 26f6f6a2b7104f1897367eabf9eca68f (DEFAULT_TABLE table=test-table [id=03bae0ddaff14f6a81cc3840f4b7d643]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:09:07.898939 24933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26f6f6a2b7104f1897367eabf9eca68f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:07.899343 25275 tablet_bootstrap.cc:492] T 26f6f6a2b7104f1897367eabf9eca68f P ad6948765ea142c58c3b13702d835ca2: Bootstrap starting.
I20260925 22:09:07.900053 25275 tablet_bootstrap.cc:654] T 26f6f6a2b7104f1897367eabf9eca68f P ad6948765ea142c58c3b13702d835ca2: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:07.900049 25276 tablet_bootstrap.cc:492] T 26f6f6a2b7104f1897367eabf9eca68f P 5aab8263b1b44d34ac9ce7e079ee7077: Bootstrap starting.
I20260925 22:09:07.900475 25275 log.cc:824] T 26f6f6a2b7104f1897367eabf9eca68f P ad6948765ea142c58c3b13702d835ca2: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:07.900676 25276 tablet_bootstrap.cc:654] T 26f6f6a2b7104f1897367eabf9eca68f P 5aab8263b1b44d34ac9ce7e079ee7077: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:07.900956 25276 log.cc:824] T 26f6f6a2b7104f1897367eabf9eca68f P 5aab8263b1b44d34ac9ce7e079ee7077: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:07.901302 25275 tablet_bootstrap.cc:492] T 26f6f6a2b7104f1897367eabf9eca68f P ad6948765ea142c58c3b13702d835ca2: No bootstrap required, opened a new log
I20260925 22:09:07.901356 25275 ts_tablet_manager.cc:1397] T 26f6f6a2b7104f1897367eabf9eca68f P ad6948765ea142c58c3b13702d835ca2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 22:09:07.901782 25276 tablet_bootstrap.cc:492] T 26f6f6a2b7104f1897367eabf9eca68f P 5aab8263b1b44d34ac9ce7e079ee7077: No bootstrap required, opened a new log
I20260925 22:09:07.901852 25276 ts_tablet_manager.cc:1397] T 26f6f6a2b7104f1897367eabf9eca68f P 5aab8263b1b44d34ac9ce7e079ee7077: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:09:07.902160 25279 tablet_bootstrap.cc:492] T 26f6f6a2b7104f1897367eabf9eca68f P 2af9c96d863f44b2b9d87dc118d7fb97: Bootstrap starting.
I20260925 22:09:07.902733 25279 tablet_bootstrap.cc:654] T 26f6f6a2b7104f1897367eabf9eca68f P 2af9c96d863f44b2b9d87dc118d7fb97: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:07.903121 25279 log.cc:824] T 26f6f6a2b7104f1897367eabf9eca68f P 2af9c96d863f44b2b9d87dc118d7fb97: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:07.903788 25279 tablet_bootstrap.cc:492] T 26f6f6a2b7104f1897367eabf9eca68f P 2af9c96d863f44b2b9d87dc118d7fb97: No bootstrap required, opened a new log
I20260925 22:09:07.903856 25279 ts_tablet_manager.cc:1397] T 26f6f6a2b7104f1897367eabf9eca68f P 2af9c96d863f44b2b9d87dc118d7fb97: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:09:07.904050 25275 raft_consensus.cc:348] T 26f6f6a2b7104f1897367eabf9eca68f P ad6948765ea142c58c3b13702d835ca2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad6948765ea142c58c3b13702d835ca2" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 39279 } } peers { permanent_uuid: "2af9c96d863f44b2b9d87dc118d7fb97" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 43653 } } peers { permanent_uuid: "5aab8263b1b44d34ac9ce7e079ee7077" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 42213 } }
I20260925 22:09:07.904228 25275 raft_consensus.cc:374] T 26f6f6a2b7104f1897367eabf9eca68f P ad6948765ea142c58c3b13702d835ca2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:07.904264 25275 raft_consensus.cc:729] T 26f6f6a2b7104f1897367eabf9eca68f P ad6948765ea142c58c3b13702d835ca2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad6948765ea142c58c3b13702d835ca2, State: Initialized, Role: FOLLOWER
I20260925 22:09:07.904407 25275 consensus_queue.cc:260] T 26f6f6a2b7104f1897367eabf9eca68f P ad6948765ea142c58c3b13702d835ca2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad6948765ea142c58c3b13702d835ca2" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 39279 } } peers { permanent_uuid: "2af9c96d863f44b2b9d87dc118d7fb97" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 43653 } } peers { permanent_uuid: "5aab8263b1b44d34ac9ce7e079ee7077" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 42213 } }
I20260925 22:09:07.904426 25276 raft_consensus.cc:348] T 26f6f6a2b7104f1897367eabf9eca68f P 5aab8263b1b44d34ac9ce7e079ee7077 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad6948765ea142c58c3b13702d835ca2" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 39279 } } peers { permanent_uuid: "2af9c96d863f44b2b9d87dc118d7fb97" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 43653 } } peers { permanent_uuid: "5aab8263b1b44d34ac9ce7e079ee7077" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 42213 } }
I20260925 22:09:07.904572 25276 raft_consensus.cc:374] T 26f6f6a2b7104f1897367eabf9eca68f P 5aab8263b1b44d34ac9ce7e079ee7077 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:07.904616 25276 raft_consensus.cc:729] T 26f6f6a2b7104f1897367eabf9eca68f P 5aab8263b1b44d34ac9ce7e079ee7077 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aab8263b1b44d34ac9ce7e079ee7077, State: Initialized, Role: FOLLOWER
I20260925 22:09:07.904685 25275 ts_tablet_manager.cc:1428] T 26f6f6a2b7104f1897367eabf9eca68f P ad6948765ea142c58c3b13702d835ca2: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260925 22:09:07.904726 25276 consensus_queue.cc:260] T 26f6f6a2b7104f1897367eabf9eca68f P 5aab8263b1b44d34ac9ce7e079ee7077 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad6948765ea142c58c3b13702d835ca2" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 39279 } } peers { permanent_uuid: "2af9c96d863f44b2b9d87dc118d7fb97" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 43653 } } peers { permanent_uuid: "5aab8263b1b44d34ac9ce7e079ee7077" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 42213 } }
I20260925 22:09:07.904806 25130 heartbeater.cc:503] Master 127.5.0.126:41681 was elected leader, sending a full tablet report...
I20260925 22:09:07.904973 25276 ts_tablet_manager.cc:1428] T 26f6f6a2b7104f1897367eabf9eca68f P 5aab8263b1b44d34ac9ce7e079ee7077: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 22:09:07.905009 25261 heartbeater.cc:503] Master 127.5.0.126:41681 was elected leader, sending a full tablet report...
I20260925 22:09:07.906440 25279 raft_consensus.cc:348] T 26f6f6a2b7104f1897367eabf9eca68f P 2af9c96d863f44b2b9d87dc118d7fb97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad6948765ea142c58c3b13702d835ca2" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 39279 } } peers { permanent_uuid: "2af9c96d863f44b2b9d87dc118d7fb97" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 43653 } } peers { permanent_uuid: "5aab8263b1b44d34ac9ce7e079ee7077" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 42213 } }
I20260925 22:09:07.906594 25279 raft_consensus.cc:374] T 26f6f6a2b7104f1897367eabf9eca68f P 2af9c96d863f44b2b9d87dc118d7fb97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:07.906630 25279 raft_consensus.cc:729] T 26f6f6a2b7104f1897367eabf9eca68f P 2af9c96d863f44b2b9d87dc118d7fb97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2af9c96d863f44b2b9d87dc118d7fb97, State: Initialized, Role: FOLLOWER
I20260925 22:09:07.906747 25279 consensus_queue.cc:260] T 26f6f6a2b7104f1897367eabf9eca68f P 2af9c96d863f44b2b9d87dc118d7fb97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad6948765ea142c58c3b13702d835ca2" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 39279 } } peers { permanent_uuid: "2af9c96d863f44b2b9d87dc118d7fb97" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 43653 } } peers { permanent_uuid: "5aab8263b1b44d34ac9ce7e079ee7077" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 42213 } }
I20260925 22:09:07.906952 25279 ts_tablet_manager.cc:1428] T 26f6f6a2b7104f1897367eabf9eca68f P 2af9c96d863f44b2b9d87dc118d7fb97: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 22:09:07.907023 24999 heartbeater.cc:503] Master 127.5.0.126:41681 was elected leader, sending a full tablet report...
I20260925 22:09:07.918977 25282 raft_consensus.cc:482] T 26f6f6a2b7104f1897367eabf9eca68f P 5aab8263b1b44d34ac9ce7e079ee7077 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:09:07.919080 25282 raft_consensus.cc:504] T 26f6f6a2b7104f1897367eabf9eca68f P 5aab8263b1b44d34ac9ce7e079ee7077 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad6948765ea142c58c3b13702d835ca2" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 39279 } } peers { permanent_uuid: "2af9c96d863f44b2b9d87dc118d7fb97" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 43653 } } peers { permanent_uuid: "5aab8263b1b44d34ac9ce7e079ee7077" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 42213 } }
I20260925 22:09:07.919421 25282 leader_election.cc:288] T 26f6f6a2b7104f1897367eabf9eca68f P 5aab8263b1b44d34ac9ce7e079ee7077 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad6948765ea142c58c3b13702d835ca2 (127.5.0.66:39279), 2af9c96d863f44b2b9d87dc118d7fb97 (127.5.0.65:43653)
I20260925 22:09:07.922314 25271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.919694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:52833 (local address 127.5.0.66:39279)
0925 22:09:07.919755 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:07.919757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:07.919768 (+ 11us) server_negotiation.cc:407] Connection header received
0925 22:09:07.919809 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:07.919809 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:07.919820 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:07.919871 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:09:07.920115 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.920663 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.921601 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.921982 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.922034 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.922042 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:07.922204 (+ 162us) server_negotiation.cc:299] Negotiation successful
0925 22:09:07.922249 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 22:09:07.922315 25263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.919609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.66:39279 (local address 127.5.0.67:52833)
0925 22:09:07.919666 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:07.919679 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:07.919745 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:07.919947 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:07.919948 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:07.919950 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:09:07.920033 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:07.920035 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.920741 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.920742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:07.921505 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:07.921507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.922067 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.922068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:07.922141 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.922144 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:09:07.922200 (+ 56us) client_negotiation.cc:240] Negotiation successful
0925 22:09:07.922245 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260925 22:09:07.922379 25274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.919803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.0.67:56947 (local address 127.5.0.65:43653)
0925 22:09:07.919864 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:07.919866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:07.919950 (+ 84us) server_negotiation.cc:407] Connection header received
0925 22:09:07.920036 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:07.920038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:07.920049 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:07.920105 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:09:07.920245 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.920687 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.921652 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.921998 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.922088 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.922097 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:07.922260 (+ 163us) server_negotiation.cc:299] Negotiation successful
0925 22:09:07.922305 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 22:09:07.922646 25284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.919739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:43653 (local address 127.5.0.67:56947)
0925 22:09:07.919875 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:07.919902 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:07.919959 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:07.920112 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:07.920113 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:07.920114 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:09:07.920177 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:07.920178 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.920794 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.920795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:07.921514 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:07.921515 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.922098 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.922099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:07.922167 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.922170 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:09:07.922529 (+ 359us) client_negotiation.cc:240] Negotiation successful
0925 22:09:07.922573 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":92,"threads_started":1}
I20260925 22:09:07.922626 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26f6f6a2b7104f1897367eabf9eca68f" candidate_uuid: "5aab8263b1b44d34ac9ce7e079ee7077" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad6948765ea142c58c3b13702d835ca2" is_pre_election: true
I20260925 22:09:07.922767 25084 raft_consensus.cc:2454] T 26f6f6a2b7104f1897367eabf9eca68f P ad6948765ea142c58c3b13702d835ca2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5aab8263b1b44d34ac9ce7e079ee7077 in term 0.
I20260925 22:09:07.922969 25149 leader_election.cc:302] T 26f6f6a2b7104f1897367eabf9eca68f P 5aab8263b1b44d34ac9ce7e079ee7077 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aab8263b1b44d34ac9ce7e079ee7077, ad6948765ea142c58c3b13702d835ca2; no voters:
I20260925 22:09:07.922967 24953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26f6f6a2b7104f1897367eabf9eca68f" candidate_uuid: "5aab8263b1b44d34ac9ce7e079ee7077" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2af9c96d863f44b2b9d87dc118d7fb97" is_pre_election: true
I20260925 22:09:07.923101 24953 raft_consensus.cc:2454] T 26f6f6a2b7104f1897367eabf9eca68f P 2af9c96d863f44b2b9d87dc118d7fb97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5aab8263b1b44d34ac9ce7e079ee7077 in term 0.
I20260925 22:09:07.923094 25282 raft_consensus.cc:2781] T 26f6f6a2b7104f1897367eabf9eca68f P 5aab8263b1b44d34ac9ce7e079ee7077 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:09:07.923143 25282 raft_consensus.cc:482] T 26f6f6a2b7104f1897367eabf9eca68f P 5aab8263b1b44d34ac9ce7e079ee7077 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:09:07.923168 25282 raft_consensus.cc:3037] T 26f6f6a2b7104f1897367eabf9eca68f P 5aab8263b1b44d34ac9ce7e079ee7077 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:07.923840 25282 raft_consensus.cc:504] T 26f6f6a2b7104f1897367eabf9eca68f P 5aab8263b1b44d34ac9ce7e079ee7077 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad6948765ea142c58c3b13702d835ca2" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 39279 } } peers { permanent_uuid: "2af9c96d863f44b2b9d87dc118d7fb97" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 43653 } } peers { permanent_uuid: "5aab8263b1b44d34ac9ce7e079ee7077" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 42213 } }
I20260925 22:09:07.924000 25282 leader_election.cc:288] T 26f6f6a2b7104f1897367eabf9eca68f P 5aab8263b1b44d34ac9ce7e079ee7077 [CANDIDATE]: Term 1 election: Requested vote from peers ad6948765ea142c58c3b13702d835ca2 (127.5.0.66:39279), 2af9c96d863f44b2b9d87dc118d7fb97 (127.5.0.65:43653)
I20260925 22:09:07.924152 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26f6f6a2b7104f1897367eabf9eca68f" candidate_uuid: "5aab8263b1b44d34ac9ce7e079ee7077" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad6948765ea142c58c3b13702d835ca2"
I20260925 22:09:07.924196 24953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26f6f6a2b7104f1897367eabf9eca68f" candidate_uuid: "5aab8263b1b44d34ac9ce7e079ee7077" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2af9c96d863f44b2b9d87dc118d7fb97"
I20260925 22:09:07.924237 25084 raft_consensus.cc:3037] T 26f6f6a2b7104f1897367eabf9eca68f P ad6948765ea142c58c3b13702d835ca2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:07.924255 24953 raft_consensus.cc:3037] T 26f6f6a2b7104f1897367eabf9eca68f P 2af9c96d863f44b2b9d87dc118d7fb97 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:07.925086 25084 raft_consensus.cc:2454] T 26f6f6a2b7104f1897367eabf9eca68f P ad6948765ea142c58c3b13702d835ca2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aab8263b1b44d34ac9ce7e079ee7077 in term 1.
I20260925 22:09:07.925086 24953 raft_consensus.cc:2454] T 26f6f6a2b7104f1897367eabf9eca68f P 2af9c96d863f44b2b9d87dc118d7fb97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aab8263b1b44d34ac9ce7e079ee7077 in term 1.
I20260925 22:09:07.925266 25149 leader_election.cc:302] T 26f6f6a2b7104f1897367eabf9eca68f P 5aab8263b1b44d34ac9ce7e079ee7077 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aab8263b1b44d34ac9ce7e079ee7077, ad6948765ea142c58c3b13702d835ca2; no voters:
I20260925 22:09:07.925364 25282 raft_consensus.cc:2781] T 26f6f6a2b7104f1897367eabf9eca68f P 5aab8263b1b44d34ac9ce7e079ee7077 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:09:07.925534 25282 raft_consensus.cc:686] T 26f6f6a2b7104f1897367eabf9eca68f P 5aab8263b1b44d34ac9ce7e079ee7077 [term 1 LEADER]: Becoming Leader. State: Replica: 5aab8263b1b44d34ac9ce7e079ee7077, State: Running, Role: LEADER
I20260925 22:09:07.925626 25282 consensus_queue.cc:237] T 26f6f6a2b7104f1897367eabf9eca68f P 5aab8263b1b44d34ac9ce7e079ee7077 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad6948765ea142c58c3b13702d835ca2" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 39279 } } peers { permanent_uuid: "2af9c96d863f44b2b9d87dc118d7fb97" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 43653 } } peers { permanent_uuid: "5aab8263b1b44d34ac9ce7e079ee7077" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 42213 } }
I20260925 22:09:07.926369 24811 catalog_manager.cc:5495] T 26f6f6a2b7104f1897367eabf9eca68f P 5aab8263b1b44d34ac9ce7e079ee7077 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5aab8263b1b44d34ac9ce7e079ee7077 (127.5.0.67). New cstate: current_term: 1 leader_uuid: "5aab8263b1b44d34ac9ce7e079ee7077" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad6948765ea142c58c3b13702d835ca2" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 39279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2af9c96d863f44b2b9d87dc118d7fb97" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 43653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aab8263b1b44d34ac9ce7e079ee7077" member_type: VOTER last_known_addr { host: "127.5.0.67" port: 42213 } health_report { overall_health: HEALTHY } } }
I20260925 22:09:07.934553 25273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.932197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35938 (local address 127.5.0.67:42213)
0925 22:09:07.932239 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:07.932241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:07.932251 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:09:07.932329 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:07.932330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:07.932346 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:07.932430 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:09:07.932571 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.933026 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.933788 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:07.933982 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:07.934054 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:07.934218 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:09:07.934316 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:09:07.934401 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:07.934443 (+ 42us) server_negotiation.cc:299] Negotiation successful
0925 22:09:07.934460 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 22:09:07.936412 24953 raft_consensus.cc:1261] T 26f6f6a2b7104f1897367eabf9eca68f P 2af9c96d863f44b2b9d87dc118d7fb97 [term 1 FOLLOWER]: Refusing update from remote peer 5aab8263b1b44d34ac9ce7e079ee7077: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:09:07.936479 25084 raft_consensus.cc:1261] T 26f6f6a2b7104f1897367eabf9eca68f P ad6948765ea142c58c3b13702d835ca2 [term 1 FOLLOWER]: Refusing update from remote peer 5aab8263b1b44d34ac9ce7e079ee7077: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:09:07.936748 25285 consensus_queue.cc:1035] T 26f6f6a2b7104f1897367eabf9eca68f P 5aab8263b1b44d34ac9ce7e079ee7077 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2af9c96d863f44b2b9d87dc118d7fb97" member_type: VOTER last_known_addr { host: "127.5.0.65" port: 43653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:09:07.936872 25282 consensus_queue.cc:1035] T 26f6f6a2b7104f1897367eabf9eca68f P 5aab8263b1b44d34ac9ce7e079ee7077 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad6948765ea142c58c3b13702d835ca2" member_type: VOTER last_known_addr { host: "127.5.0.66" port: 39279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:09:07.939849 25289 mvcc.cc:205] Tried to move back new op lower bound from 7333372509944807424 to 7333372509904072704. Current Snapshot: MvccSnapshot[applied={T|T < 7333372509944807424}]
I20260925 22:09:07.939951 25290 mvcc.cc:205] Tried to move back new op lower bound from 7333372509944807424 to 7333372509904072704. Current Snapshot: MvccSnapshot[applied={T|T < 7333372509944807424}]
I20260925 22:09:07.941403 24811 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:09:07.941507 24811 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:09:07.942323 24811 catalog_manager.cc:5782] T 00000000000000000000000000000000 P db55b0f4c00741ac98b69f235b0e831f: Sending DeleteTablet for 3 replicas of tablet 26f6f6a2b7104f1897367eabf9eca68f
I20260925 22:09:07.942704 25064 tablet_service.cc:1514] Processing DeleteTablet for tablet 26f6f6a2b7104f1897367eabf9eca68f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:09:07 UTC) from {username='slave'} at 127.0.0.1:59074
I20260925 22:09:07.942761 24933 tablet_service.cc:1514] Processing DeleteTablet for tablet 26f6f6a2b7104f1897367eabf9eca68f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:09:07 UTC) from {username='slave'} at 127.0.0.1:60688
I20260925 22:09:07.942981 25195 tablet_service.cc:1514] Processing DeleteTablet for tablet 26f6f6a2b7104f1897367eabf9eca68f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:09:07 UTC) from {username='slave'} at 127.0.0.1:35934
I20260925 22:09:07.943015 25297 tablet_replica.cc:330] T 26f6f6a2b7104f1897367eabf9eca68f P ad6948765ea142c58c3b13702d835ca2: stopping tablet replica
I20260925 22:09:07.943122 25297 raft_consensus.cc:2229] T 26f6f6a2b7104f1897367eabf9eca68f P ad6948765ea142c58c3b13702d835ca2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:09:07.943220 25297 raft_consensus.cc:2258] T 26f6f6a2b7104f1897367eabf9eca68f P ad6948765ea142c58c3b13702d835ca2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:09:07.943301 25299 tablet_replica.cc:330] T 26f6f6a2b7104f1897367eabf9eca68f P 5aab8263b1b44d34ac9ce7e079ee7077: stopping tablet replica
I20260925 22:09:07.943441 25299 raft_consensus.cc:2229] T 26f6f6a2b7104f1897367eabf9eca68f P 5aab8263b1b44d34ac9ce7e079ee7077 [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:09:07.943531 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 24871
I20260925 22:09:07.943655 25297 ts_tablet_manager.cc:1905] T 26f6f6a2b7104f1897367eabf9eca68f P ad6948765ea142c58c3b13702d835ca2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:09:07.944314 25299 raft_consensus.cc:2258] T 26f6f6a2b7104f1897367eabf9eca68f P 5aab8263b1b44d34ac9ce7e079ee7077 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:09:07.944831 25299 ts_tablet_manager.cc:1905] T 26f6f6a2b7104f1897367eabf9eca68f P 5aab8263b1b44d34ac9ce7e079ee7077: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:09:07.945885 25297 ts_tablet_manager.cc:1918] T 26f6f6a2b7104f1897367eabf9eca68f P ad6948765ea142c58c3b13702d835ca2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:09:07.945961 25297 log.cc:1196] T 26f6f6a2b7104f1897367eabf9eca68f P ad6948765ea142c58c3b13702d835ca2: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-1/wal/wals/26f6f6a2b7104f1897367eabf9eca68f
I20260925 22:09:07.946223 25297 ts_tablet_manager.cc:1939] T 26f6f6a2b7104f1897367eabf9eca68f P ad6948765ea142c58c3b13702d835ca2: Deleting consensus metadata
I20260925 22:09:07.946628 24796 catalog_manager.cc:4841] TS ad6948765ea142c58c3b13702d835ca2 (127.5.0.66:39279): tablet 26f6f6a2b7104f1897367eabf9eca68f (table test-table [id=03bae0ddaff14f6a81cc3840f4b7d643]) successfully deleted
I20260925 22:09:07.947422 25299 ts_tablet_manager.cc:1918] T 26f6f6a2b7104f1897367eabf9eca68f P 5aab8263b1b44d34ac9ce7e079ee7077: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:09:07.947497 25299 log.cc:1196] T 26f6f6a2b7104f1897367eabf9eca68f P 5aab8263b1b44d34ac9ce7e079ee7077: Deleting WAL directory at /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373908987421-5121-0/minicluster-data/ts-2/wal/wals/26f6f6a2b7104f1897367eabf9eca68f
I20260925 22:09:07.947758 25299 ts_tablet_manager.cc:1939] T 26f6f6a2b7104f1897367eabf9eca68f P 5aab8263b1b44d34ac9ce7e079ee7077: Deleting consensus metadata
W20260925 22:09:07.951973 24796 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.0.65:43653 (error 108)
I20260925 22:09:07.952037 24798 catalog_manager.cc:4841] TS 5aab8263b1b44d34ac9ce7e079ee7077 (127.5.0.67:42213): tablet 26f6f6a2b7104f1897367eabf9eca68f (table test-table [id=03bae0ddaff14f6a81cc3840f4b7d643]) successfully deleted
I20260925 22:09:07.952425 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 25002
I20260925 22:09:07.952549 25272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:07.952324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.65:43653 (local address 127.0.0.1:60696)
0925 22:09:07.952468 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:07.952510 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.65:43653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
W20260925 22:09:07.953599 24796 catalog_manager.cc:4584] TS 2af9c96d863f44b2b9d87dc118d7fb97 (127.5.0.65:43653): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 26f6f6a2b7104f1897367eabf9eca68f: Network error: Client connection negotiation failed: client connection to 127.5.0.65:43653: connect: Connection refused (error 111)
I20260925 22:09:07.957279 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 25133
I20260925 22:09:07.963090 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 24780
2026-09-25T22:09:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (656 ms)
[----------] 6 tests from EncryptionPolicyTest (8017 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:09:08 dist-test-slave-7mqr krb5kdc[25303](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:09:08 dist-test-slave-7mqr krb5kdc[25303](info): set up 2 sockets
Sep 25 22:09:08 dist-test-slave-7mqr krb5kdc[25303](info): commencing operation
krb5kdc: starting...
W20260925 22:09:09.024263 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:09:09 dist-test-slave-7mqr krb5kdc[25303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:09:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:09:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 22:09:09.146679 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42927
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42927
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:09:09.227223 25316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:09.227430 25316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:09.227452 25316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:09.228663 25316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:09:09.228708 25316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:09.228721 25316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:09.228734 25316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:09:09.228788 25316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:09:09.230226 25316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42927
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:09.230482 25316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:09.230716 25316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:09.235301 25321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:09.235301 25324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:09.235383 25322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:09.235761 25316 server_base.cc:1034] running on GCE node
I20260925 22:09:09.236044 25316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:09.236320 25316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:09.237486 25316 hybrid_clock.cc:648] HybridClock initialized: now 1790374149237474 us; error 30 us; skew 500 ppm
Sep 25 22:09:09 dist-test-slave-7mqr krb5kdc[25303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374149, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:09.239830 25316 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 22:09:09.240269 25316 webserver.cc:460] Webserver started at http://127.0.0.1:40851/ using document root <none> and password file <none>
I20260925 22:09:09.240549 25316 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:09.240621 25316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:09.240797 25316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:09.241822 25316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "4b77d27822a84bccafc7a23d0ccd9387"
format_stamp: "Formatted at 2026-09-25 22:09:09 on dist-test-slave-7mqr"
server_key: "228e3bc21155f3a97d9db69c79bb2b05"
server_key_iv: "7401df480554fffd9686bbe906fc4fee"
server_key_version: "encryptionkey@0"
I20260925 22:09:09.242170 25316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "4b77d27822a84bccafc7a23d0ccd9387"
format_stamp: "Formatted at 2026-09-25 22:09:09 on dist-test-slave-7mqr"
server_key: "228e3bc21155f3a97d9db69c79bb2b05"
server_key_iv: "7401df480554fffd9686bbe906fc4fee"
server_key_version: "encryptionkey@0"
I20260925 22:09:09.243476 25316 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 22:09:09.244243 25331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:09.244410 25316 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:09.244545 25316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "4b77d27822a84bccafc7a23d0ccd9387"
format_stamp: "Formatted at 2026-09-25 22:09:09 on dist-test-slave-7mqr"
server_key: "228e3bc21155f3a97d9db69c79bb2b05"
server_key_iv: "7401df480554fffd9686bbe906fc4fee"
server_key_version: "encryptionkey@0"
I20260925 22:09:09.244619 25316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:09.251041 25316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:09.252813 25316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:09.252972 25316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:09.256706 25316 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42927
I20260925 22:09:09.256735 25383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42927 every 8 connection(s)
I20260925 22:09:09.257068 25316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:09:09.258044 25384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:09.260666 25384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b77d27822a84bccafc7a23d0ccd9387: Bootstrap starting.
I20260925 22:09:09.261142 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 25316
I20260925 22:09:09.261181 25384 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b77d27822a84bccafc7a23d0ccd9387: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:09.261276 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:09:09.261476 5121 external_mini_cluster.cc:1428] Setting key 08a411e83b7fd98357b79cb65391012f
I20260925 22:09:09.261484 25384 log.cc:824] T 00000000000000000000000000000000 P 4b77d27822a84bccafc7a23d0ccd9387: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:09.262234 25384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b77d27822a84bccafc7a23d0ccd9387: No bootstrap required, opened a new log
I20260925 22:09:09.264809 25384 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b77d27822a84bccafc7a23d0ccd9387 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b77d27822a84bccafc7a23d0ccd9387" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42927 } }
I20260925 22:09:09.264974 25384 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b77d27822a84bccafc7a23d0ccd9387 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:09.265018 25384 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b77d27822a84bccafc7a23d0ccd9387 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b77d27822a84bccafc7a23d0ccd9387, State: Initialized, Role: FOLLOWER
I20260925 22:09:09.265133 25384 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b77d27822a84bccafc7a23d0ccd9387 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b77d27822a84bccafc7a23d0ccd9387" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42927 } }
I20260925 22:09:09.265206 25384 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b77d27822a84bccafc7a23d0ccd9387 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:09:09.265235 25384 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b77d27822a84bccafc7a23d0ccd9387 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:09:09.265278 25384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b77d27822a84bccafc7a23d0ccd9387 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:09.265885 25384 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b77d27822a84bccafc7a23d0ccd9387 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b77d27822a84bccafc7a23d0ccd9387" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42927 } }
I20260925 22:09:09.265971 25384 leader_election.cc:302] T 00000000000000000000000000000000 P 4b77d27822a84bccafc7a23d0ccd9387 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b77d27822a84bccafc7a23d0ccd9387; no voters:
I20260925 22:09:09.266197 25384 leader_election.cc:288] T 00000000000000000000000000000000 P 4b77d27822a84bccafc7a23d0ccd9387 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:09:09.266220 25389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b77d27822a84bccafc7a23d0ccd9387 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:09:09.266358 25389 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b77d27822a84bccafc7a23d0ccd9387 [term 1 LEADER]: Becoming Leader. State: Replica: 4b77d27822a84bccafc7a23d0ccd9387, State: Running, Role: LEADER
I20260925 22:09:09.266441 25389 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b77d27822a84bccafc7a23d0ccd9387 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b77d27822a84bccafc7a23d0ccd9387" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42927 } }
I20260925 22:09:09.266497 25384 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b77d27822a84bccafc7a23d0ccd9387 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:09:09.266741 25389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b77d27822a84bccafc7a23d0ccd9387 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b77d27822a84bccafc7a23d0ccd9387" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b77d27822a84bccafc7a23d0ccd9387" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42927 } } }
I20260925 22:09:09.266809 25389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b77d27822a84bccafc7a23d0ccd9387 [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:09.266935 25389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b77d27822a84bccafc7a23d0ccd9387 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b77d27822a84bccafc7a23d0ccd9387. Latest consensus state: current_term: 1 leader_uuid: "4b77d27822a84bccafc7a23d0ccd9387" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b77d27822a84bccafc7a23d0ccd9387" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42927 } } }
I20260925 22:09:09.266993 25389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b77d27822a84bccafc7a23d0ccd9387 [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:09.267112 25393 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:09:09.267632 25393 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 25 22:09:09 dist-test-slave-7mqr krb5kdc[25303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 22:09:09.269237 25393 catalog_manager.cc:1347] Generated new cluster ID: 0666ae46c3ad45eaa2e5dcb98a0a212d
I20260925 22:09:09.269291 25393 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:09:09.272874 25388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:09.261988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57735 (local address 127.0.0.1:42927)
0925 22:09:09.262146 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:09.262154 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:09.262247 (+ 93us) server_negotiation.cc:407] Connection header received
0925 22:09:09.262760 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:09.262767 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:09.262812 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:09.262922 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:09.263707 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:09.264637 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:09.265702 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:09.266199 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:09.266320 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:09.269267 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:09.269271 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:09.269271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:09.269283 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:09.271197 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:09.271662 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:09.271665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:09.271665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:09.271759 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:09.272025 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:09.272025 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:09.272025 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:09.272291 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:09.272443 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:09.272588 (+ 145us) server_negotiation.cc:299] Negotiation successful
0925 22:09:09.272686 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":71,"threads_started":1}
I20260925 22:09:09.273931 5121 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:09:09.282784 25388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:09.275385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59347 (local address 127.0.0.1:42927)
0925 22:09:09.275443 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:09.275446 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:09.275558 (+ 112us) server_negotiation.cc:407] Connection header received
0925 22:09:09.275583 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:09.275584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:09.275593 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:09.275662 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:09.276435 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:09.277188 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:09.278153 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:09.278499 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:09.278581 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:09.279912 (+ 1331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:09.279913 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:09.279914 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:09.279921 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:09.281558 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:09.281961 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:09.281962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:09.281962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:09.282019 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:09.282262 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:09.282263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:09.282263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:09.282385 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:09.282487 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:09.282628 (+ 141us) server_negotiation.cc:299] Negotiation successful
0925 22:09:09.282691 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 22:09:09.283493 25393 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:09:09.284080 25393 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:09:09.287588 25393 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b77d27822a84bccafc7a23d0ccd9387: Generated new TSK 0
I20260925 22:09:09.287758 25393 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:09:09.292047 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 25316
W20260925 22:09:09.296793 25301 connection.cc:535] client connection to 127.0.0.1:42927 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42927): Connection reset by peer (error 104)
2026-09-25T22:09:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1327 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:09:09 dist-test-slave-7mqr krb5kdc[25415](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:09:09 dist-test-slave-7mqr krb5kdc[25415](info): set up 2 sockets
Sep 25 22:09:09 dist-test-slave-7mqr krb5kdc[25415](info): commencing operation
krb5kdc: starting...
W20260925 22:09:10.327961 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:09:10 dist-test-slave-7mqr krb5kdc[25415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:09:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:09:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 22:09:10.446892 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45187
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45187
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:09:10.528074 25428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:10.528273 25428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:10.528294 25428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:10.529459 25428 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:09:10.529505 25428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:10.529518 25428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:10.529531 25428 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:09:10.529583 25428 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:09:10.531093 25428 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45187
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:10.531334 25428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:10.531579 25428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:10.536249 25436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:10.536412 25428 server_base.cc:1034] running on GCE node
W20260925 22:09:10.536293 25433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:10.536249 25434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:10.536728 25428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:10.536959 25428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:10.538132 25428 hybrid_clock.cc:648] HybridClock initialized: now 1790374150538121 us; error 30 us; skew 500 ppm
Sep 25 22:09:10 dist-test-slave-7mqr krb5kdc[25415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374150, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:10.546209 25428 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 22:09:10.546691 25428 webserver.cc:460] Webserver started at http://127.0.0.1:35291/ using document root <none> and password file <none>
I20260925 22:09:10.546949 25428 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:10.546996 25428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:10.547135 25428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:10.548295 25428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "2b9527bb825644b0a7fa2107f4636d1d"
format_stamp: "Formatted at 2026-09-25 22:09:10 on dist-test-slave-7mqr"
server_key: "a9df15d52b148930ad8716585366455f"
server_key_iv: "ec98724b168a70edc2d694f6d5659575"
server_key_version: "encryptionkey@0"
I20260925 22:09:10.548652 25428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "2b9527bb825644b0a7fa2107f4636d1d"
format_stamp: "Formatted at 2026-09-25 22:09:10 on dist-test-slave-7mqr"
server_key: "a9df15d52b148930ad8716585366455f"
server_key_iv: "ec98724b168a70edc2d694f6d5659575"
server_key_version: "encryptionkey@0"
I20260925 22:09:10.550011 25428 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:09:10.550729 25443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:10.550900 25428 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:09:10.551014 25428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "2b9527bb825644b0a7fa2107f4636d1d"
format_stamp: "Formatted at 2026-09-25 22:09:10 on dist-test-slave-7mqr"
server_key: "a9df15d52b148930ad8716585366455f"
server_key_iv: "ec98724b168a70edc2d694f6d5659575"
server_key_version: "encryptionkey@0"
I20260925 22:09:10.551091 25428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:10.561197 25428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:10.562952 25428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:10.563115 25428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:10.566918 25428 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45187
I20260925 22:09:10.566983 25495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45187 every 8 connection(s)
I20260925 22:09:10.567332 25428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:09:10.568336 25496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:10.570465 25496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b9527bb825644b0a7fa2107f4636d1d: Bootstrap starting.
I20260925 22:09:10.571033 25496 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b9527bb825644b0a7fa2107f4636d1d: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:10.571309 25496 log.cc:824] T 00000000000000000000000000000000 P 2b9527bb825644b0a7fa2107f4636d1d: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:10.571640 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 25428
I20260925 22:09:10.571761 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:09:10.571998 5121 external_mini_cluster.cc:1428] Setting key 83f53fff013ea31a87ad3c72794c6f75
I20260925 22:09:10.572139 25496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b9527bb825644b0a7fa2107f4636d1d: No bootstrap required, opened a new log
I20260925 22:09:10.574997 25496 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b9527bb825644b0a7fa2107f4636d1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9527bb825644b0a7fa2107f4636d1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45187 } }
I20260925 22:09:10.575165 25496 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b9527bb825644b0a7fa2107f4636d1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:10.575196 25496 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b9527bb825644b0a7fa2107f4636d1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b9527bb825644b0a7fa2107f4636d1d, State: Initialized, Role: FOLLOWER
I20260925 22:09:10.575318 25496 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b9527bb825644b0a7fa2107f4636d1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9527bb825644b0a7fa2107f4636d1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45187 } }
I20260925 22:09:10.575389 25496 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b9527bb825644b0a7fa2107f4636d1d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:09:10.575434 25496 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b9527bb825644b0a7fa2107f4636d1d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:09:10.575488 25496 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b9527bb825644b0a7fa2107f4636d1d [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:10.576098 25496 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b9527bb825644b0a7fa2107f4636d1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9527bb825644b0a7fa2107f4636d1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45187 } }
I20260925 22:09:10.576212 25496 leader_election.cc:302] T 00000000000000000000000000000000 P 2b9527bb825644b0a7fa2107f4636d1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b9527bb825644b0a7fa2107f4636d1d; no voters:
I20260925 22:09:10.576418 25496 leader_election.cc:288] T 00000000000000000000000000000000 P 2b9527bb825644b0a7fa2107f4636d1d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:09:10.576448 25501 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b9527bb825644b0a7fa2107f4636d1d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:09:10.576625 25501 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b9527bb825644b0a7fa2107f4636d1d [term 1 LEADER]: Becoming Leader. State: Replica: 2b9527bb825644b0a7fa2107f4636d1d, State: Running, Role: LEADER
I20260925 22:09:10.576700 25496 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b9527bb825644b0a7fa2107f4636d1d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:09:10.576756 25501 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b9527bb825644b0a7fa2107f4636d1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9527bb825644b0a7fa2107f4636d1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45187 } }
I20260925 22:09:10.577069 25501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b9527bb825644b0a7fa2107f4636d1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b9527bb825644b0a7fa2107f4636d1d. Latest consensus state: current_term: 1 leader_uuid: "2b9527bb825644b0a7fa2107f4636d1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9527bb825644b0a7fa2107f4636d1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45187 } } }
I20260925 22:09:10.577288 25501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b9527bb825644b0a7fa2107f4636d1d [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:10.577221 25503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b9527bb825644b0a7fa2107f4636d1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b9527bb825644b0a7fa2107f4636d1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9527bb825644b0a7fa2107f4636d1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45187 } } }
I20260925 22:09:10.577450 25503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b9527bb825644b0a7fa2107f4636d1d [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:10.577606 25509 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:09:10.578082 25509 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:09:10.578850 25500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:10.572503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45125 (local address 127.0.0.1:45187)
0925 22:09:10.572685 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:10.572695 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:10.572728 (+ 33us) server_negotiation.cc:407] Connection header received
0925 22:09:10.573519 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:10.573528 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:10.573590 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:10.573737 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:10.573943 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:10.575349 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:10.576264 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:10.576897 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:10.577071 (+ 174us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:10.577843 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:10.577847 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:10.577848 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:10.578083 (+ 235us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:10.578145 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:10.578162 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:10.578254 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:10.578607 (+ 353us) server_negotiation.cc:299] Negotiation successful
0925 22:09:10.578636 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":76,"threads_started":1}
I20260925 22:09:10.579587 25509 catalog_manager.cc:1347] Generated new cluster ID: e8d7107f144640468503463f5574fa5a
I20260925 22:09:10.579638 25509 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:09:10.580037 5121 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:09:10.584779 25500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:10.581359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39339 (local address 127.0.0.1:45187)
0925 22:09:10.581395 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:10.581397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:10.581509 (+ 112us) server_negotiation.cc:407] Connection header received
0925 22:09:10.581537 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:10.581537 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:10.581546 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:10.581615 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:10.581765 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:10.582481 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:10.583435 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:10.583746 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:10.583820 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:10.584496 (+ 676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:10.584498 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:10.584498 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:10.584503 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:10.584518 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:10.584523 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:10.584596 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:10.584674 (+ 78us) server_negotiation.cc:299] Negotiation successful
0925 22:09:10.584686 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 22:09:10.589318 25509 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:09:10.589982 25509 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:09:10.600083 25509 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b9527bb825644b0a7fa2107f4636d1d: Generated new TSK 0
I20260925 22:09:10.600275 25509 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:09:10.603322 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 25428
W20260925 22:09:10.607461 25413 connection.cc:535] client connection to 127.0.0.1:45187 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45187): Connection reset by peer (error 104)
2026-09-25T22:09:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1310 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:09:10 dist-test-slave-7mqr krb5kdc[25527](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:09:10 dist-test-slave-7mqr krb5kdc[25527](info): set up 2 sockets
Sep 25 22:09:10 dist-test-slave-7mqr krb5kdc[25527](info): commencing operation
krb5kdc: starting...
W20260925 22:09:11.639842 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:09:11 dist-test-slave-7mqr krb5kdc[25527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:09:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:09:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 22:09:11.758564 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41495
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:09:11.840430 25540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:11.840660 25540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:11.840700 25540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:11.841848 25540 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:09:11.841915 25540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:11.841939 25540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:11.841962 25540 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:09:11.841990 25540 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:09:11.843539 25540 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41495
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:11.843816 25540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:11.844063 25540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:11.848968 25546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:11.849018 25548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:11.848968 25545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:11.849154 25540 server_base.cc:1034] running on GCE node
I20260925 22:09:11.849423 25540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:11.849733 25540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:11.850917 25540 hybrid_clock.cc:648] HybridClock initialized: now 1790374151850896 us; error 37 us; skew 500 ppm
Sep 25 22:09:11 dist-test-slave-7mqr krb5kdc[25527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374151, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:11.853542 25540 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 22:09:11.853976 25540 webserver.cc:460] Webserver started at http://127.0.0.1:43901/ using document root <none> and password file <none>
I20260925 22:09:11.854219 25540 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:11.854285 25540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:11.854442 25540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:11.855489 25540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "e4b782b1ccc94984b43ec0a96d47f97f"
format_stamp: "Formatted at 2026-09-25 22:09:11 on dist-test-slave-7mqr"
server_key: "edc0d82fe467d0b3ba8617f606cc0a01"
server_key_iv: "e14975515a54383a08c4e49eecea7baa"
server_key_version: "encryptionkey@0"
I20260925 22:09:11.855849 25540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "e4b782b1ccc94984b43ec0a96d47f97f"
format_stamp: "Formatted at 2026-09-25 22:09:11 on dist-test-slave-7mqr"
server_key: "edc0d82fe467d0b3ba8617f606cc0a01"
server_key_iv: "e14975515a54383a08c4e49eecea7baa"
server_key_version: "encryptionkey@0"
I20260925 22:09:11.857240 25540 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:09:11.858021 25555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:11.858320 25540 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 22:09:11.858426 25540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "e4b782b1ccc94984b43ec0a96d47f97f"
format_stamp: "Formatted at 2026-09-25 22:09:11 on dist-test-slave-7mqr"
server_key: "edc0d82fe467d0b3ba8617f606cc0a01"
server_key_iv: "e14975515a54383a08c4e49eecea7baa"
server_key_version: "encryptionkey@0"
I20260925 22:09:11.858537 25540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:11.861857 25540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:11.864151 25540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:11.864310 25540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:11.868340 25540 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41495
I20260925 22:09:11.868376 25607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41495 every 8 connection(s)
I20260925 22:09:11.868723 25540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:09:11.869652 25608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:11.871871 25608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4b782b1ccc94984b43ec0a96d47f97f: Bootstrap starting.
I20260925 22:09:11.872723 25608 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4b782b1ccc94984b43ec0a96d47f97f: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:11.873123 25608 log.cc:824] T 00000000000000000000000000000000 P e4b782b1ccc94984b43ec0a96d47f97f: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:11.873238 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 25540
I20260925 22:09:11.873366 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:09:11.873548 5121 external_mini_cluster.cc:1428] Setting key c7eaf205ce4dfa9990ac3ddc2ce6202b
I20260925 22:09:11.873952 25608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4b782b1ccc94984b43ec0a96d47f97f: No bootstrap required, opened a new log
I20260925 22:09:11.875648 25608 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4b782b1ccc94984b43ec0a96d47f97f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4b782b1ccc94984b43ec0a96d47f97f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41495 } }
I20260925 22:09:11.875808 25608 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4b782b1ccc94984b43ec0a96d47f97f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:11.875855 25608 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4b782b1ccc94984b43ec0a96d47f97f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4b782b1ccc94984b43ec0a96d47f97f, State: Initialized, Role: FOLLOWER
I20260925 22:09:11.876006 25608 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4b782b1ccc94984b43ec0a96d47f97f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4b782b1ccc94984b43ec0a96d47f97f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41495 } }
I20260925 22:09:11.876109 25608 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4b782b1ccc94984b43ec0a96d47f97f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:09:11.876149 25608 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4b782b1ccc94984b43ec0a96d47f97f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:09:11.876190 25608 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4b782b1ccc94984b43ec0a96d47f97f [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:11.877000 25612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:11.874091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43641 (local address 127.0.0.1:41495)
0925 22:09:11.874294 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:11.874304 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:11.874336 (+ 32us) server_negotiation.cc:407] Connection header received
0925 22:09:11.875637 (+ 1301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:11.875645 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:11.875675 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:11.875856 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:11.876576 (+ 720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:11.876579 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:11.876580 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:11.876584 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:11.876629 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:11.876640 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:11.876716 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:11.876777 (+ 61us) server_negotiation.cc:299] Negotiation successful
0925 22:09:11.876819 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":92,"threads_started":1}
I20260925 22:09:11.877311 25608 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4b782b1ccc94984b43ec0a96d47f97f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4b782b1ccc94984b43ec0a96d47f97f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41495 } }
I20260925 22:09:11.877447 25608 leader_election.cc:302] T 00000000000000000000000000000000 P e4b782b1ccc94984b43ec0a96d47f97f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4b782b1ccc94984b43ec0a96d47f97f; no voters:
I20260925 22:09:11.877619 25608 leader_election.cc:288] T 00000000000000000000000000000000 P e4b782b1ccc94984b43ec0a96d47f97f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:09:11.877668 25613 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4b782b1ccc94984b43ec0a96d47f97f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:09:11.877882 25608 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4b782b1ccc94984b43ec0a96d47f97f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:09:11.877971 5121 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:09:11.877935 25613 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4b782b1ccc94984b43ec0a96d47f97f [term 1 LEADER]: Becoming Leader. State: Replica: e4b782b1ccc94984b43ec0a96d47f97f, State: Running, Role: LEADER
I20260925 22:09:11.878307 25613 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4b782b1ccc94984b43ec0a96d47f97f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4b782b1ccc94984b43ec0a96d47f97f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41495 } }
I20260925 22:09:11.878721 25615 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4b782b1ccc94984b43ec0a96d47f97f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4b782b1ccc94984b43ec0a96d47f97f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4b782b1ccc94984b43ec0a96d47f97f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41495 } } }
I20260925 22:09:11.878806 25615 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4b782b1ccc94984b43ec0a96d47f97f [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:11.879217 25614 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4b782b1ccc94984b43ec0a96d47f97f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4b782b1ccc94984b43ec0a96d47f97f. Latest consensus state: current_term: 1 leader_uuid: "e4b782b1ccc94984b43ec0a96d47f97f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4b782b1ccc94984b43ec0a96d47f97f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41495 } } }
I20260925 22:09:11.879343 25614 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4b782b1ccc94984b43ec0a96d47f97f [sys.catalog]: This master's current role is: LEADER
W20260925 22:09:11.879925 25633 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e4b782b1ccc94984b43ec0a96d47f97f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:09:11.879981 25633 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:09:11.880031 25622 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:09:11.880205 25622 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:09:11.881215 25612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:11.879820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37989 (local address 127.0.0.1:41495)
0925 22:09:11.879856 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:11.879858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:11.880067 (+ 209us) server_negotiation.cc:407] Connection header received
0925 22:09:11.880209 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:11.880210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:11.880215 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:11.880278 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:11.880959 (+ 681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:11.880960 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:11.880960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:11.880964 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:11.880979 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:11.880985 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:11.881047 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:11.881108 (+ 61us) server_negotiation.cc:299] Negotiation successful
0925 22:09:11.881120 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 22:09:11.882126 25622 catalog_manager.cc:1347] Generated new cluster ID: a1374341dd62447f9beaea84ce979ae8
I20260925 22:09:11.882196 25622 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:09:11.905859 25622 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:09:11.906083 25622 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:09:11.910535 25622 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4b782b1ccc94984b43ec0a96d47f97f: Generated new TSK 0
I20260925 22:09:11.910751 25622 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:09:11.914848 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 25540
2026-09-25T22:09:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1311 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:09:11 dist-test-slave-7mqr krb5kdc[25639](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:09:11 dist-test-slave-7mqr krb5kdc[25639](info): set up 2 sockets
Sep 25 22:09:11 dist-test-slave-7mqr krb5kdc[25639](info): commencing operation
krb5kdc: starting...
W20260925 22:09:12.960000 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:09:13 dist-test-slave-7mqr krb5kdc[25639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:09:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:09:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 22:09:13.078841 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37467
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37467
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:09:13.158206 25652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:13.158412 25652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:13.158434 25652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:13.159626 25652 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:09:13.159670 25652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:13.159684 25652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:13.159698 25652 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:09:13.159752 25652 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:09:13.161199 25652 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37467
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:13.161450 25652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:13.161686 25652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:13.166378 25660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:13.166364 25657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:13.166412 25658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:13.166595 25652 server_base.cc:1034] running on GCE node
I20260925 22:09:13.166814 25652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:13.167043 25652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:13.168179 25652 hybrid_clock.cc:648] HybridClock initialized: now 1790374153168167 us; error 26 us; skew 500 ppm
Sep 25 22:09:13 dist-test-slave-7mqr krb5kdc[25639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374153, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:13.170677 25652 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 22:09:13.171131 25652 webserver.cc:460] Webserver started at http://127.0.0.1:44231/ using document root <none> and password file <none>
I20260925 22:09:13.171355 25652 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:13.171422 25652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:13.171579 25652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:13.172683 25652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "570d71fcdc444bccbb6e006bda57989e"
format_stamp: "Formatted at 2026-09-25 22:09:13 on dist-test-slave-7mqr"
server_key: "802b70db3bde64b221f8566f2607622e"
server_key_iv: "5a4944ba8d06c1adf645515a2f66a0be"
server_key_version: "encryptionkey@0"
I20260925 22:09:13.173041 25652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "570d71fcdc444bccbb6e006bda57989e"
format_stamp: "Formatted at 2026-09-25 22:09:13 on dist-test-slave-7mqr"
server_key: "802b70db3bde64b221f8566f2607622e"
server_key_iv: "5a4944ba8d06c1adf645515a2f66a0be"
server_key_version: "encryptionkey@0"
I20260925 22:09:13.174306 25652 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:09:13.175041 25667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:13.175238 25652 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:09:13.175351 25652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "570d71fcdc444bccbb6e006bda57989e"
format_stamp: "Formatted at 2026-09-25 22:09:13 on dist-test-slave-7mqr"
server_key: "802b70db3bde64b221f8566f2607622e"
server_key_iv: "5a4944ba8d06c1adf645515a2f66a0be"
server_key_version: "encryptionkey@0"
I20260925 22:09:13.175428 25652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:13.185500 25652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:13.187717 25652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:13.187875 25652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:13.191483 25652 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37467
I20260925 22:09:13.191543 25719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37467 every 8 connection(s)
I20260925 22:09:13.191876 25652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:09:13.192864 25720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:13.193238 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 25652
I20260925 22:09:13.193353 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:09:13.193543 5121 external_mini_cluster.cc:1428] Setting key aa015af111f44e980bd27c450c2d4804
I20260925 22:09:13.195842 25720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 570d71fcdc444bccbb6e006bda57989e: Bootstrap starting.
I20260925 22:09:13.196437 25720 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 570d71fcdc444bccbb6e006bda57989e: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:13.196713 25720 log.cc:824] T 00000000000000000000000000000000 P 570d71fcdc444bccbb6e006bda57989e: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:13.197449 25720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 570d71fcdc444bccbb6e006bda57989e: No bootstrap required, opened a new log
I20260925 22:09:13.199960 25720 raft_consensus.cc:348] T 00000000000000000000000000000000 P 570d71fcdc444bccbb6e006bda57989e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "570d71fcdc444bccbb6e006bda57989e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37467 } }
I20260925 22:09:13.200099 25720 raft_consensus.cc:374] T 00000000000000000000000000000000 P 570d71fcdc444bccbb6e006bda57989e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:13.200122 25720 raft_consensus.cc:729] T 00000000000000000000000000000000 P 570d71fcdc444bccbb6e006bda57989e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 570d71fcdc444bccbb6e006bda57989e, State: Initialized, Role: FOLLOWER
I20260925 22:09:13.200260 25720 consensus_queue.cc:260] T 00000000000000000000000000000000 P 570d71fcdc444bccbb6e006bda57989e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "570d71fcdc444bccbb6e006bda57989e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37467 } }
I20260925 22:09:13.200318 25720 raft_consensus.cc:388] T 00000000000000000000000000000000 P 570d71fcdc444bccbb6e006bda57989e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:09:13.200340 25720 raft_consensus.cc:482] T 00000000000000000000000000000000 P 570d71fcdc444bccbb6e006bda57989e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:09:13.200404 25720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 570d71fcdc444bccbb6e006bda57989e [term 0 FOLLOWER]: Advancing to term 1
Sep 25 22:09:13 dist-test-slave-7mqr krb5kdc[25639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 22:09:13.201084 25720 raft_consensus.cc:504] T 00000000000000000000000000000000 P 570d71fcdc444bccbb6e006bda57989e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "570d71fcdc444bccbb6e006bda57989e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37467 } }
I20260925 22:09:13.201202 25720 leader_election.cc:302] T 00000000000000000000000000000000 P 570d71fcdc444bccbb6e006bda57989e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 570d71fcdc444bccbb6e006bda57989e; no voters:
I20260925 22:09:13.201380 25720 leader_election.cc:288] T 00000000000000000000000000000000 P 570d71fcdc444bccbb6e006bda57989e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:09:13.201407 25725 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 570d71fcdc444bccbb6e006bda57989e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:09:13.201530 25725 raft_consensus.cc:686] T 00000000000000000000000000000000 P 570d71fcdc444bccbb6e006bda57989e [term 1 LEADER]: Becoming Leader. State: Replica: 570d71fcdc444bccbb6e006bda57989e, State: Running, Role: LEADER
I20260925 22:09:13.201637 25720 sys_catalog.cc:564] T 00000000000000000000000000000000 P 570d71fcdc444bccbb6e006bda57989e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:09:13.201671 25725 consensus_queue.cc:237] T 00000000000000000000000000000000 P 570d71fcdc444bccbb6e006bda57989e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "570d71fcdc444bccbb6e006bda57989e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37467 } }
I20260925 22:09:13.202068 25727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 570d71fcdc444bccbb6e006bda57989e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "570d71fcdc444bccbb6e006bda57989e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "570d71fcdc444bccbb6e006bda57989e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37467 } } }
I20260925 22:09:13.202212 25727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 570d71fcdc444bccbb6e006bda57989e [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:13.202068 25726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 570d71fcdc444bccbb6e006bda57989e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 570d71fcdc444bccbb6e006bda57989e. Latest consensus state: current_term: 1 leader_uuid: "570d71fcdc444bccbb6e006bda57989e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "570d71fcdc444bccbb6e006bda57989e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37467 } } }
I20260925 22:09:13.203218 25726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 570d71fcdc444bccbb6e006bda57989e [sys.catalog]: This master's current role is: LEADER
W20260925 22:09:13.203532 25740 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 570d71fcdc444bccbb6e006bda57989e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:09:13.203617 25740 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:09:13.203714 25734 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:09:13.203878 25734 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:09:13.204653 25723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:13.194032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51749 (local address 127.0.0.1:37467)
0925 22:09:13.194205 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:13.194213 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:13.194322 (+ 109us) server_negotiation.cc:407] Connection header received
0925 22:09:13.194749 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:13.194756 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:13.194802 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:13.194912 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:13.195722 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:13.196694 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:13.197707 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:13.198104 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:13.198183 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:13.200954 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:13.200958 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:13.200958 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:13.200984 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:13.202889 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:13.203274 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:13.203277 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:13.203278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:13.203384 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:13.203606 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:13.203607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:13.203608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:13.203925 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:13.204070 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:13.204293 (+ 223us) server_negotiation.cc:299] Negotiation successful
0925 22:09:13.204393 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":83,"threads_started":1}
I20260925 22:09:13.205621 5121 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:09:13.208854 25734 catalog_manager.cc:1347] Generated new cluster ID: f5ea1dcd4bbf4ebc94edc87d106149b1
I20260925 22:09:13.208906 25734 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:09:13.213917 25723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:13.207076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45887 (local address 127.0.0.1:37467)
0925 22:09:13.207117 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:13.207120 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:13.207172 (+ 52us) server_negotiation.cc:407] Connection header received
0925 22:09:13.207229 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:13.207231 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:13.207240 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:13.207307 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:13.207746 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:13.208521 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:13.209514 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:13.209830 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:13.209904 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:13.211147 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:13.211148 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:13.211148 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:13.211155 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:13.212765 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:13.213168 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:13.213169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:13.213170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:13.213227 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:13.213418 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:13.213418 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:13.213419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:13.213554 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:13.213649 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:13.213755 (+ 106us) server_negotiation.cc:299] Negotiation successful
0925 22:09:13.213831 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 22:09:13.223460 25734 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:09:13.224027 25734 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:09:13.234190 25734 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 570d71fcdc444bccbb6e006bda57989e: Generated new TSK 0
I20260925 22:09:13.234377 25734 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:09:13.239711 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 25652
2026-09-25T22:09:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1324 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:09:13 dist-test-slave-7mqr krb5kdc[25751](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:09:13 dist-test-slave-7mqr krb5kdc[25751](info): set up 2 sockets
Sep 25 22:09:13 dist-test-slave-7mqr krb5kdc[25751](info): commencing operation
krb5kdc: starting...
W20260925 22:09:14.268014 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:09:14 dist-test-slave-7mqr krb5kdc[25751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:09:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:09:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 22:09:14.387104 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34925
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34925
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:09:14.466231 25764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:14.466501 25764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:14.466542 25764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:14.467648 25764 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:09:14.467729 25764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:14.467752 25764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:14.467774 25764 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:09:14.467801 25764 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:09:14.469255 25764 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:14.469525 25764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:14.469738 25764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 22:09:14.474491 25764 server_base.cc:1034] running on GCE node
W20260925 22:09:14.474463 25769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:14.474500 25770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:14.474613 25772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:14.474870 25764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:14.475190 25764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:14.476348 25764 hybrid_clock.cc:648] HybridClock initialized: now 1790374154476334 us; error 40 us; skew 500 ppm
Sep 25 22:09:14 dist-test-slave-7mqr krb5kdc[25751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374154, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:14.478698 25764 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 22:09:14.479164 25764 webserver.cc:460] Webserver started at http://127.0.0.1:43909/ using document root <none> and password file <none>
I20260925 22:09:14.479387 25764 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:14.479455 25764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:14.479610 25764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:14.480731 25764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "c1d19ad413a448f1be303de1fa62ebd2"
format_stamp: "Formatted at 2026-09-25 22:09:14 on dist-test-slave-7mqr"
server_key: "2742d46c19ec07c7848eeff25f86989f"
server_key_iv: "b8a772a59c020033775c52d44807dfce"
server_key_version: "encryptionkey@0"
I20260925 22:09:14.481117 25764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "c1d19ad413a448f1be303de1fa62ebd2"
format_stamp: "Formatted at 2026-09-25 22:09:14 on dist-test-slave-7mqr"
server_key: "2742d46c19ec07c7848eeff25f86989f"
server_key_iv: "b8a772a59c020033775c52d44807dfce"
server_key_version: "encryptionkey@0"
I20260925 22:09:14.482487 25764 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 22:09:14.483310 25779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:14.483522 25764 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:09:14.483646 25764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "c1d19ad413a448f1be303de1fa62ebd2"
format_stamp: "Formatted at 2026-09-25 22:09:14 on dist-test-slave-7mqr"
server_key: "2742d46c19ec07c7848eeff25f86989f"
server_key_iv: "b8a772a59c020033775c52d44807dfce"
server_key_version: "encryptionkey@0"
I20260925 22:09:14.483736 25764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:14.507408 25764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:14.509852 25764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:14.510025 25764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:14.514140 25764 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34925
I20260925 22:09:14.514194 25831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34925 every 8 connection(s)
I20260925 22:09:14.514573 25764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:09:14.515527 25832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:14.517810 25832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1d19ad413a448f1be303de1fa62ebd2: Bootstrap starting.
I20260925 22:09:14.518337 25832 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1d19ad413a448f1be303de1fa62ebd2: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:14.518571 25832 log.cc:824] T 00000000000000000000000000000000 P c1d19ad413a448f1be303de1fa62ebd2: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:14.519193 25832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1d19ad413a448f1be303de1fa62ebd2: No bootstrap required, opened a new log
I20260925 22:09:14.520812 25832 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1d19ad413a448f1be303de1fa62ebd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1d19ad413a448f1be303de1fa62ebd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34925 } }
I20260925 22:09:14.520948 25832 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1d19ad413a448f1be303de1fa62ebd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:14.520972 25832 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1d19ad413a448f1be303de1fa62ebd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1d19ad413a448f1be303de1fa62ebd2, State: Initialized, Role: FOLLOWER
I20260925 22:09:14.521075 25832 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1d19ad413a448f1be303de1fa62ebd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1d19ad413a448f1be303de1fa62ebd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34925 } }
I20260925 22:09:14.521126 25832 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1d19ad413a448f1be303de1fa62ebd2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:09:14.521147 25832 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1d19ad413a448f1be303de1fa62ebd2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:09:14.521215 25832 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1d19ad413a448f1be303de1fa62ebd2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:14.521682 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 25764
I20260925 22:09:14.521770 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:09:14.521764 25832 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1d19ad413a448f1be303de1fa62ebd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1d19ad413a448f1be303de1fa62ebd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34925 } }
I20260925 22:09:14.521896 25832 leader_election.cc:302] T 00000000000000000000000000000000 P c1d19ad413a448f1be303de1fa62ebd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1d19ad413a448f1be303de1fa62ebd2; no voters:
I20260925 22:09:14.521986 5121 external_mini_cluster.cc:1428] Setting key 0d68fe4633c62dedaea4c5d875acb2b5
I20260925 22:09:14.522089 25832 leader_election.cc:288] T 00000000000000000000000000000000 P c1d19ad413a448f1be303de1fa62ebd2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:09:14.522140 25835 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1d19ad413a448f1be303de1fa62ebd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:09:14.522346 25835 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1d19ad413a448f1be303de1fa62ebd2 [term 1 LEADER]: Becoming Leader. State: Replica: c1d19ad413a448f1be303de1fa62ebd2, State: Running, Role: LEADER
I20260925 22:09:14.522487 25835 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1d19ad413a448f1be303de1fa62ebd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1d19ad413a448f1be303de1fa62ebd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34925 } }
I20260925 22:09:14.522373 25832 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1d19ad413a448f1be303de1fa62ebd2 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:09:14.522810 25835 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1d19ad413a448f1be303de1fa62ebd2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1d19ad413a448f1be303de1fa62ebd2. Latest consensus state: current_term: 1 leader_uuid: "c1d19ad413a448f1be303de1fa62ebd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1d19ad413a448f1be303de1fa62ebd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34925 } } }
I20260925 22:09:14.522926 25835 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1d19ad413a448f1be303de1fa62ebd2 [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:14.523082 25837 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1d19ad413a448f1be303de1fa62ebd2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1d19ad413a448f1be303de1fa62ebd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1d19ad413a448f1be303de1fa62ebd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34925 } } }
I20260925 22:09:14.523169 25837 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1d19ad413a448f1be303de1fa62ebd2 [sys.catalog]: This master's current role is: LEADER
W20260925 22:09:14.524770 25853 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c1d19ad413a448f1be303de1fa62ebd2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:09:14.524829 25853 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:09:14.524878 25841 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:09:14.525013 25841 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:09:14.526347 25841 catalog_manager.cc:1347] Generated new cluster ID: 05bcb46c1cac4414922fd6fc30a49a62
I20260925 22:09:14.526415 25841 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:09:14.527740 25839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:14.522572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34969 (local address 127.0.0.1:34925)
0925 22:09:14.522730 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:14.522741 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:14.522916 (+ 175us) server_negotiation.cc:407] Connection header received
0925 22:09:14.523050 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:14.523059 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:14.523121 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:14.523257 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:14.523620 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:14.524819 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:14.525698 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:14.526186 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:14.526321 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:14.527074 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:14.527078 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:14.527079 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:14.527266 (+ 187us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:14.527296 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:14.527306 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:14.527379 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:14.527453 (+ 74us) server_negotiation.cc:299] Negotiation successful
0925 22:09:14.527526 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":73,"threads_started":1}
I20260925 22:09:14.528592 5121 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:09:14.534896 25839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:14.529833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52395 (local address 127.0.0.1:34925)
0925 22:09:14.529869 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:14.529870 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:14.532237 (+ 2367us) server_negotiation.cc:407] Connection header received
0925 22:09:14.532307 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:14.532308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:14.532319 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:14.532374 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:14.532504 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:14.532984 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:14.533698 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:14.533911 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:14.533986 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:14.534640 (+ 654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:14.534641 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:14.534641 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:14.534647 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:14.534660 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:14.534665 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:14.534719 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:14.534778 (+ 59us) server_negotiation.cc:299] Negotiation successful
0925 22:09:14.534799 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 22:09:14.536731 25841 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:09:14.537444 25841 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:09:14.544389 25841 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1d19ad413a448f1be303de1fa62ebd2: Generated new TSK 0
I20260925 22:09:14.544618 25841 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:09:14.548424 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 25764
2026-09-25T22:09:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1309 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:09:14 dist-test-slave-7mqr krb5kdc[25863](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:09:14 dist-test-slave-7mqr krb5kdc[25863](info): set up 2 sockets
Sep 25 22:09:14 dist-test-slave-7mqr krb5kdc[25863](info): commencing operation
krb5kdc: starting...
W20260925 22:09:15.580031 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:09:15 dist-test-slave-7mqr krb5kdc[25863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:09:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:09:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 22:09:15.698793 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36875
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:09:15.781051 25876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:15.781286 25876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:15.781322 25876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:15.782454 25876 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:09:15.782534 25876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:15.782562 25876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:15.782593 25876 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:09:15.782616 25876 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:09:15.784056 25876 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36875
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:15.784327 25876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:15.784560 25876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:15.789186 25881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:15.789235 25882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:15.789186 25884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:15.789546 25876 server_base.cc:1034] running on GCE node
I20260925 22:09:15.789804 25876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:15.790088 25876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:15.791249 25876 hybrid_clock.cc:648] HybridClock initialized: now 1790374155791214 us; error 58 us; skew 500 ppm
Sep 25 22:09:15 dist-test-slave-7mqr krb5kdc[25863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374155, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:15.799064 25876 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 22:09:15.799633 25876 webserver.cc:460] Webserver started at http://127.0.0.1:38287/ using document root <none> and password file <none>
I20260925 22:09:15.799949 25876 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:15.800034 25876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:15.800203 25876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:15.801332 25876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "e174f68beb8c44cc975c5ad40e1fcde9"
format_stamp: "Formatted at 2026-09-25 22:09:15 on dist-test-slave-7mqr"
server_key: "0aea6e07d1aa26eaef4afa38f4565202"
server_key_iv: "8cb774a00fc2b47d3278d379383aea8e"
server_key_version: "encryptionkey@0"
I20260925 22:09:15.801728 25876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "e174f68beb8c44cc975c5ad40e1fcde9"
format_stamp: "Formatted at 2026-09-25 22:09:15 on dist-test-slave-7mqr"
server_key: "0aea6e07d1aa26eaef4afa38f4565202"
server_key_iv: "8cb774a00fc2b47d3278d379383aea8e"
server_key_version: "encryptionkey@0"
I20260925 22:09:15.803381 25876 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:09:15.804205 25891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:15.804391 25876 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:09:15.804512 25876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "e174f68beb8c44cc975c5ad40e1fcde9"
format_stamp: "Formatted at 2026-09-25 22:09:15 on dist-test-slave-7mqr"
server_key: "0aea6e07d1aa26eaef4afa38f4565202"
server_key_iv: "8cb774a00fc2b47d3278d379383aea8e"
server_key_version: "encryptionkey@0"
I20260925 22:09:15.804597 25876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:15.808465 25876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:15.810892 25876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:15.811070 25876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:15.815454 25876 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36875
I20260925 22:09:15.815482 25943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36875 every 8 connection(s)
I20260925 22:09:15.815824 25876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:09:15.816802 25944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:15.819056 25944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e174f68beb8c44cc975c5ad40e1fcde9: Bootstrap starting.
I20260925 22:09:15.819622 25944 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e174f68beb8c44cc975c5ad40e1fcde9: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:15.819985 25944 log.cc:824] T 00000000000000000000000000000000 P e174f68beb8c44cc975c5ad40e1fcde9: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:15.820719 25944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e174f68beb8c44cc975c5ad40e1fcde9: No bootstrap required, opened a new log
I20260925 22:09:15.822305 25944 raft_consensus.cc:348] T 00000000000000000000000000000000 P e174f68beb8c44cc975c5ad40e1fcde9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e174f68beb8c44cc975c5ad40e1fcde9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36875 } }
I20260925 22:09:15.822443 25944 raft_consensus.cc:374] T 00000000000000000000000000000000 P e174f68beb8c44cc975c5ad40e1fcde9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:15.822491 25944 raft_consensus.cc:729] T 00000000000000000000000000000000 P e174f68beb8c44cc975c5ad40e1fcde9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e174f68beb8c44cc975c5ad40e1fcde9, State: Initialized, Role: FOLLOWER
I20260925 22:09:15.822656 25944 consensus_queue.cc:260] T 00000000000000000000000000000000 P e174f68beb8c44cc975c5ad40e1fcde9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e174f68beb8c44cc975c5ad40e1fcde9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36875 } }
I20260925 22:09:15.822719 25944 raft_consensus.cc:388] T 00000000000000000000000000000000 P e174f68beb8c44cc975c5ad40e1fcde9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:09:15.822760 25944 raft_consensus.cc:482] T 00000000000000000000000000000000 P e174f68beb8c44cc975c5ad40e1fcde9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:09:15.822809 25944 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e174f68beb8c44cc975c5ad40e1fcde9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:15.823355 25944 raft_consensus.cc:504] T 00000000000000000000000000000000 P e174f68beb8c44cc975c5ad40e1fcde9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e174f68beb8c44cc975c5ad40e1fcde9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36875 } }
I20260925 22:09:15.823469 25944 leader_election.cc:302] T 00000000000000000000000000000000 P e174f68beb8c44cc975c5ad40e1fcde9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e174f68beb8c44cc975c5ad40e1fcde9; no voters:
I20260925 22:09:15.823658 25944 leader_election.cc:288] T 00000000000000000000000000000000 P e174f68beb8c44cc975c5ad40e1fcde9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:09:15.823727 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 25876
I20260925 22:09:15.823724 25947 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e174f68beb8c44cc975c5ad40e1fcde9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:09:15.823843 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:09:15.823884 25947 raft_consensus.cc:686] T 00000000000000000000000000000000 P e174f68beb8c44cc975c5ad40e1fcde9 [term 1 LEADER]: Becoming Leader. State: Replica: e174f68beb8c44cc975c5ad40e1fcde9, State: Running, Role: LEADER
I20260925 22:09:15.823972 25944 sys_catalog.cc:564] T 00000000000000000000000000000000 P e174f68beb8c44cc975c5ad40e1fcde9 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:09:15.823982 25947 consensus_queue.cc:237] T 00000000000000000000000000000000 P e174f68beb8c44cc975c5ad40e1fcde9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e174f68beb8c44cc975c5ad40e1fcde9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36875 } }
I20260925 22:09:15.824141 5121 external_mini_cluster.cc:1428] Setting key 20c0442dfb800cc0c560d012de7c7828
I20260925 22:09:15.824438 25949 sys_catalog.cc:455] T 00000000000000000000000000000000 P e174f68beb8c44cc975c5ad40e1fcde9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e174f68beb8c44cc975c5ad40e1fcde9. Latest consensus state: current_term: 1 leader_uuid: "e174f68beb8c44cc975c5ad40e1fcde9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e174f68beb8c44cc975c5ad40e1fcde9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36875 } } }
I20260925 22:09:15.824532 25949 sys_catalog.cc:458] T 00000000000000000000000000000000 P e174f68beb8c44cc975c5ad40e1fcde9 [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:15.824571 25948 sys_catalog.cc:455] T 00000000000000000000000000000000 P e174f68beb8c44cc975c5ad40e1fcde9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e174f68beb8c44cc975c5ad40e1fcde9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e174f68beb8c44cc975c5ad40e1fcde9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36875 } } }
I20260925 22:09:15.824630 25948 sys_catalog.cc:458] T 00000000000000000000000000000000 P e174f68beb8c44cc975c5ad40e1fcde9 [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:15.824841 25955 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:09:15.825598 25955 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:09:15.826740 25958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:15.824796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60127 (local address 127.0.0.1:36875)
0925 22:09:15.825160 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:15.825171 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:15.825205 (+ 34us) server_negotiation.cc:407] Connection header received
0925 22:09:15.825327 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:15.825334 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:15.825365 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:15.825856 (+ 491us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:15.826283 (+ 427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:15.826287 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:15.826287 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:15.826291 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:15.826337 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:15.826351 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:15.826447 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:15.826515 (+ 68us) server_negotiation.cc:299] Negotiation successful
0925 22:09:15.826560 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":64,"threads_started":1}
I20260925 22:09:15.827590 5121 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:09:15.827996 25955 catalog_manager.cc:1347] Generated new cluster ID: 291bff861ca14845adbb12b63b3d7a76
I20260925 22:09:15.828063 25955 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:09:15.835402 25955 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:09:15.835561 25955 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:09:15.836217 25958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:15.832218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54779 (local address 127.0.0.1:36875)
0925 22:09:15.832255 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:15.832257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:15.835158 (+ 2901us) server_negotiation.cc:407] Connection header received
0925 22:09:15.835213 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:15.835214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:15.835219 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:15.835268 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:15.835999 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:15.836000 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:15.836000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:15.836006 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:15.836023 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:15.836028 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:15.836092 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:15.836144 (+ 52us) server_negotiation.cc:299] Negotiation successful
0925 22:09:15.836157 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:09:15.840091 25955 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e174f68beb8c44cc975c5ad40e1fcde9: Generated new TSK 0
I20260925 22:09:15.840267 25955 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:09:15.845116 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 25876
2026-09-25T22:09:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1297 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:09:15 dist-test-slave-7mqr krb5kdc[25975](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:09:15 dist-test-slave-7mqr krb5kdc[25975](info): set up 2 sockets
Sep 25 22:09:15 dist-test-slave-7mqr krb5kdc[25975](info): commencing operation
krb5kdc: starting...
W20260925 22:09:16.884514 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:09:16 dist-test-slave-7mqr krb5kdc[25975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:09:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:09:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 22:09:17.002646 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37635
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:09:17.083163 25988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:17.083403 25988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:17.083442 25988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:17.084648 25988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:09:17.084725 25988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:17.084749 25988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:17.084775 25988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:09:17.084797 25988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:09:17.086282 25988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:17.086575 25988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:17.086799 25988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:17.091457 25993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:17.091434 25994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:17.091461 25988 server_base.cc:1034] running on GCE node
W20260925 22:09:17.091431 25996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:17.091996 25988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:17.092298 25988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:17.093458 25988 hybrid_clock.cc:648] HybridClock initialized: now 1790374157093451 us; error 36 us; skew 500 ppm
Sep 25 22:09:17 dist-test-slave-7mqr krb5kdc[25975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374157, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:17.095877 25988 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 22:09:17.096360 25988 webserver.cc:460] Webserver started at http://127.0.0.1:34389/ using document root <none> and password file <none>
I20260925 22:09:17.096602 25988 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:17.096669 25988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:17.096822 25988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:17.097849 25988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "a9c99eb2b41c40b5a90d16e4489a6c56"
format_stamp: "Formatted at 2026-09-25 22:09:17 on dist-test-slave-7mqr"
server_key: "6b8d17d2eea915ecbbb6e4a6fcd2a467"
server_key_iv: "f7338d909292ec10a4a5ac47d5105c13"
server_key_version: "encryptionkey@0"
I20260925 22:09:17.098201 25988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "a9c99eb2b41c40b5a90d16e4489a6c56"
format_stamp: "Formatted at 2026-09-25 22:09:17 on dist-test-slave-7mqr"
server_key: "6b8d17d2eea915ecbbb6e4a6fcd2a467"
server_key_iv: "f7338d909292ec10a4a5ac47d5105c13"
server_key_version: "encryptionkey@0"
I20260925 22:09:17.099535 25988 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:09:17.100338 26003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:17.100529 25988 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 22:09:17.100634 25988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "a9c99eb2b41c40b5a90d16e4489a6c56"
format_stamp: "Formatted at 2026-09-25 22:09:17 on dist-test-slave-7mqr"
server_key: "6b8d17d2eea915ecbbb6e4a6fcd2a467"
server_key_iv: "f7338d909292ec10a4a5ac47d5105c13"
server_key_version: "encryptionkey@0"
I20260925 22:09:17.100708 25988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:17.116485 25988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:17.118386 25988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:17.118548 25988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:17.122335 25988 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37635
I20260925 22:09:17.122383 26055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37635 every 8 connection(s)
I20260925 22:09:17.122752 25988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:09:17.123706 26056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:17.125967 26056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9c99eb2b41c40b5a90d16e4489a6c56: Bootstrap starting.
I20260925 22:09:17.126556 26056 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9c99eb2b41c40b5a90d16e4489a6c56: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:17.126832 26056 log.cc:824] T 00000000000000000000000000000000 P a9c99eb2b41c40b5a90d16e4489a6c56: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:17.127525 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 25988
I20260925 22:09:17.127641 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:09:17.127702 26056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9c99eb2b41c40b5a90d16e4489a6c56: No bootstrap required, opened a new log
I20260925 22:09:17.127847 5121 external_mini_cluster.cc:1428] Setting key 41a73df8c4833fc6919cce8cd6f88e4d
I20260925 22:09:17.129863 26056 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9c99eb2b41c40b5a90d16e4489a6c56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c99eb2b41c40b5a90d16e4489a6c56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37635 } }
I20260925 22:09:17.130020 26056 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9c99eb2b41c40b5a90d16e4489a6c56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:17.130069 26056 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9c99eb2b41c40b5a90d16e4489a6c56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9c99eb2b41c40b5a90d16e4489a6c56, State: Initialized, Role: FOLLOWER
I20260925 22:09:17.130237 26056 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9c99eb2b41c40b5a90d16e4489a6c56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c99eb2b41c40b5a90d16e4489a6c56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37635 } }
I20260925 22:09:17.130331 26056 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9c99eb2b41c40b5a90d16e4489a6c56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:09:17.130374 26056 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9c99eb2b41c40b5a90d16e4489a6c56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:09:17.130430 26056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9c99eb2b41c40b5a90d16e4489a6c56 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:17.130970 26056 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9c99eb2b41c40b5a90d16e4489a6c56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c99eb2b41c40b5a90d16e4489a6c56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37635 } }
I20260925 22:09:17.131088 26056 leader_election.cc:302] T 00000000000000000000000000000000 P a9c99eb2b41c40b5a90d16e4489a6c56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9c99eb2b41c40b5a90d16e4489a6c56; no voters:
I20260925 22:09:17.131263 26056 leader_election.cc:288] T 00000000000000000000000000000000 P a9c99eb2b41c40b5a90d16e4489a6c56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:09:17.131299 26061 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9c99eb2b41c40b5a90d16e4489a6c56 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:09:17.131410 26061 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9c99eb2b41c40b5a90d16e4489a6c56 [term 1 LEADER]: Becoming Leader. State: Replica: a9c99eb2b41c40b5a90d16e4489a6c56, State: Running, Role: LEADER
I20260925 22:09:17.131531 26056 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9c99eb2b41c40b5a90d16e4489a6c56 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:09:17.131542 26061 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9c99eb2b41c40b5a90d16e4489a6c56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c99eb2b41c40b5a90d16e4489a6c56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37635 } }
I20260925 22:09:17.132098 26062 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9c99eb2b41c40b5a90d16e4489a6c56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9c99eb2b41c40b5a90d16e4489a6c56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c99eb2b41c40b5a90d16e4489a6c56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37635 } } }
I20260925 22:09:17.132179 26062 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9c99eb2b41c40b5a90d16e4489a6c56 [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:17.132387 26063 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9c99eb2b41c40b5a90d16e4489a6c56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9c99eb2b41c40b5a90d16e4489a6c56. Latest consensus state: current_term: 1 leader_uuid: "a9c99eb2b41c40b5a90d16e4489a6c56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c99eb2b41c40b5a90d16e4489a6c56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37635 } } }
I20260925 22:09:17.132721 26063 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9c99eb2b41c40b5a90d16e4489a6c56 [sys.catalog]: This master's current role is: LEADER
W20260925 22:09:17.133329 26077 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a9c99eb2b41c40b5a90d16e4489a6c56: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:09:17.133430 26077 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:09:17.133498 26070 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:09:17.133661 26070 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:09:17.135159 26070 catalog_manager.cc:1347] Generated new cluster ID: 12f3e55212064a548c850b83a63e1a36
I20260925 22:09:17.135210 26070 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:09:17 dist-test-slave-7mqr krb5kdc[25975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 22:09:17.139650 26060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:17.128454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.212:59873 (local address 127.0.0.1:37635)
0925 22:09:17.128635 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:17.128643 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:17.128667 (+ 24us) server_negotiation.cc:407] Connection header received
0925 22:09:17.129166 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:17.129172 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:17.129220 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:17.129367 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:17.130013 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:17.131374 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:17.132493 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:17.133202 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:17.133322 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:17.135991 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:17.135995 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:17.135995 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:17.136006 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:17.137968 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:17.138398 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:17.138401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:17.138401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:17.138510 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:17.138743 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:17.138743 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:17.138744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:17.139066 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:17.139240 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:17.139339 (+ 99us) server_negotiation.cc:299] Negotiation successful
0925 22:09:17.139439 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":77,"threads_started":1}
I20260925 22:09:17.140640 5121 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:09:17.149138 26060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:17.141999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.212:43501 (local address 127.0.0.1:37635)
0925 22:09:17.142040 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:17.142042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:17.142142 (+ 100us) server_negotiation.cc:407] Connection header received
0925 22:09:17.142200 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:17.142201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:17.142210 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:17.142263 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:17.142866 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:17.143575 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:17.144619 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:17.144926 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:17.144984 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:17.146374 (+ 1390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:17.146375 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:17.146375 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:17.146382 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:17.147944 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:17.148348 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:17.148349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:17.148349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:17.148401 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:17.148622 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:17.148623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:17.148623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:17.148733 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:17.148821 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:17.148985 (+ 164us) server_negotiation.cc:299] Negotiation successful
0925 22:09:17.149054 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 22:09:17.174878 26070 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:09:17.175663 26070 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:09:17.179092 26070 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9c99eb2b41c40b5a90d16e4489a6c56: Generated new TSK 0
I20260925 22:09:17.179302 26070 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:09:17.187105 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 25988
W20260925 22:09:17.191102 25973 connection.cc:535] client connection to 127.0.0.1:37635 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37635): Connection reset by peer (error 104)
2026-09-25T22:09:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1340 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:09:17 dist-test-slave-7mqr krb5kdc[26087](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:09:17 dist-test-slave-7mqr krb5kdc[26087](info): set up 2 sockets
Sep 25 22:09:17 dist-test-slave-7mqr krb5kdc[26087](info): commencing operation
krb5kdc: starting...
W20260925 22:09:18.232043 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:09:18 dist-test-slave-7mqr krb5kdc[26087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374158, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:09:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:09:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 22:09:18.350590 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33943
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33943
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:09:18.431418 26100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:18.431672 26100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:18.431710 26100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:18.432888 26100 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:09:18.432960 26100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:18.432984 26100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:18.433013 26100 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:09:18.433058 26100 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:09:18.434553 26100 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33943
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:18.434827 26100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:18.435055 26100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:18.439917 26105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:18.439942 26106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:18.440065 26100 server_base.cc:1034] running on GCE node
W20260925 22:09:18.439939 26108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:18.440439 26100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:18.440732 26100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:18.441883 26100 hybrid_clock.cc:648] HybridClock initialized: now 1790374158441858 us; error 41 us; skew 500 ppm
Sep 25 22:09:18 dist-test-slave-7mqr krb5kdc[26087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374158, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:18.444365 26100 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 22:09:18.444792 26100 webserver.cc:460] Webserver started at http://127.0.0.1:41251/ using document root <none> and password file <none>
I20260925 22:09:18.445020 26100 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:18.445092 26100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:18.445248 26100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:18.446292 26100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "a0edfe24c9ff4d658e2a5a017476985f"
format_stamp: "Formatted at 2026-09-25 22:09:18 on dist-test-slave-7mqr"
server_key: "e0b85be071272f95bed06ad43ce53b13"
server_key_iv: "2d2d2462097203b8adc52ab5535f7daf"
server_key_version: "encryptionkey@0"
I20260925 22:09:18.446666 26100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "a0edfe24c9ff4d658e2a5a017476985f"
format_stamp: "Formatted at 2026-09-25 22:09:18 on dist-test-slave-7mqr"
server_key: "e0b85be071272f95bed06ad43ce53b13"
server_key_iv: "2d2d2462097203b8adc52ab5535f7daf"
server_key_version: "encryptionkey@0"
I20260925 22:09:18.448025 26100 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 22:09:18.448791 26115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:18.448990 26100 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:09:18.449100 26100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "a0edfe24c9ff4d658e2a5a017476985f"
format_stamp: "Formatted at 2026-09-25 22:09:18 on dist-test-slave-7mqr"
server_key: "e0b85be071272f95bed06ad43ce53b13"
server_key_iv: "2d2d2462097203b8adc52ab5535f7daf"
server_key_version: "encryptionkey@0"
I20260925 22:09:18.449187 26100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:18.473771 26100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:18.475775 26100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:18.476013 26100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:18.480120 26100 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33943
I20260925 22:09:18.480157 26167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33943 every 8 connection(s)
I20260925 22:09:18.480506 26100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:09:18.481475 26168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:18.483721 26168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0edfe24c9ff4d658e2a5a017476985f: Bootstrap starting.
I20260925 22:09:18.484305 26168 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0edfe24c9ff4d658e2a5a017476985f: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:18.484591 26168 log.cc:824] T 00000000000000000000000000000000 P a0edfe24c9ff4d658e2a5a017476985f: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:18.485252 26168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0edfe24c9ff4d658e2a5a017476985f: No bootstrap required, opened a new log
I20260925 22:09:18.485589 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 26100
I20260925 22:09:18.485716 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:09:18.485940 5121 external_mini_cluster.cc:1428] Setting key ca9271ca5b0d05bf94fa40fe16cf1139
I20260925 22:09:18.487509 26168 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0edfe24c9ff4d658e2a5a017476985f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0edfe24c9ff4d658e2a5a017476985f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33943 } }
I20260925 22:09:18.487645 26168 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0edfe24c9ff4d658e2a5a017476985f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:18.487668 26168 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0edfe24c9ff4d658e2a5a017476985f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0edfe24c9ff4d658e2a5a017476985f, State: Initialized, Role: FOLLOWER
I20260925 22:09:18.487805 26168 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0edfe24c9ff4d658e2a5a017476985f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0edfe24c9ff4d658e2a5a017476985f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33943 } }
I20260925 22:09:18.487865 26168 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0edfe24c9ff4d658e2a5a017476985f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:09:18.487931 26168 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0edfe24c9ff4d658e2a5a017476985f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:09:18.487986 26168 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0edfe24c9ff4d658e2a5a017476985f [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:18.488512 26168 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0edfe24c9ff4d658e2a5a017476985f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0edfe24c9ff4d658e2a5a017476985f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33943 } }
I20260925 22:09:18.488626 26168 leader_election.cc:302] T 00000000000000000000000000000000 P a0edfe24c9ff4d658e2a5a017476985f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0edfe24c9ff4d658e2a5a017476985f; no voters:
I20260925 22:09:18.488814 26168 leader_election.cc:288] T 00000000000000000000000000000000 P a0edfe24c9ff4d658e2a5a017476985f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:09:18.488838 26173 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0edfe24c9ff4d658e2a5a017476985f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:09:18.488958 26173 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0edfe24c9ff4d658e2a5a017476985f [term 1 LEADER]: Becoming Leader. State: Replica: a0edfe24c9ff4d658e2a5a017476985f, State: Running, Role: LEADER
I20260925 22:09:18.489039 26173 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0edfe24c9ff4d658e2a5a017476985f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0edfe24c9ff4d658e2a5a017476985f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33943 } }
I20260925 22:09:18.489081 26168 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0edfe24c9ff4d658e2a5a017476985f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:09:18.489360 26173 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0edfe24c9ff4d658e2a5a017476985f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0edfe24c9ff4d658e2a5a017476985f. Latest consensus state: current_term: 1 leader_uuid: "a0edfe24c9ff4d658e2a5a017476985f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0edfe24c9ff4d658e2a5a017476985f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33943 } } }
I20260925 22:09:18.489372 26174 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0edfe24c9ff4d658e2a5a017476985f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0edfe24c9ff4d658e2a5a017476985f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0edfe24c9ff4d658e2a5a017476985f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33943 } } }
I20260925 22:09:18.489437 26173 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0edfe24c9ff4d658e2a5a017476985f [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:18.489468 26174 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0edfe24c9ff4d658e2a5a017476985f [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:18.490221 26178 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:09:18.490731 26178 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:09:18.492487 26172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:18.486451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.212:53813 (local address 127.0.0.1:33943)
0925 22:09:18.486655 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:18.486665 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:18.486699 (+ 34us) server_negotiation.cc:407] Connection header received
0925 22:09:18.487147 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:18.487153 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:18.487201 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:18.487341 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:18.487591 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:18.489043 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:18.490199 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:18.490839 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:18.491274 (+ 435us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:18.491701 (+ 427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:18.491705 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:18.491705 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:18.491925 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:18.491965 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:18.491980 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:18.492067 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:18.492230 (+ 163us) server_negotiation.cc:299] Negotiation successful
0925 22:09:18.492264 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":77,"threads_started":1}
I20260925 22:09:18.492910 26178 catalog_manager.cc:1347] Generated new cluster ID: a3294c3c59734507af543327b3178475
I20260925 22:09:18.492971 26178 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:09:18.493481 5121 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:09:18.499804 26197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:18.495048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.212:46667 (local address 127.0.0.1:33943)
0925 22:09:18.495160 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:18.495162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:18.495169 (+ 7us) server_negotiation.cc:407] Connection header received
0925 22:09:18.495186 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:18.495187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:18.495195 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:18.495258 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:18.495427 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:18.496201 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:18.497305 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:18.497610 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:18.498694 (+ 1084us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:18.499418 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:18.499419 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:18.499419 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:18.499423 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:18.499444 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:18.499449 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:18.499507 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:18.499604 (+ 97us) server_negotiation.cc:299] Negotiation successful
0925 22:09:18.499615 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":62,"threads_started":1}
I20260925 22:09:18.511431 26178 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:09:18.512202 26178 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:09:18.523106 26178 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0edfe24c9ff4d658e2a5a017476985f: Generated new TSK 0
I20260925 22:09:18.523317 26178 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:09:18.532869 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 26100
W20260925 22:09:18.537253 26085 connection.cc:535] client connection to 127.0.0.1:33943 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33943): Connection reset by peer (error 104)
2026-09-25T22:09:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1346 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:09:18 dist-test-slave-7mqr krb5kdc[26200](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:09:18 dist-test-slave-7mqr krb5kdc[26200](info): set up 2 sockets
Sep 25 22:09:18 dist-test-slave-7mqr krb5kdc[26200](info): commencing operation
krb5kdc: starting...
W20260925 22:09:19.571847 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:09:19 dist-test-slave-7mqr krb5kdc[26200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:09:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:09:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 22:09:19.683561 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39889
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:09:19.763556 26213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:19.763762 26213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:19.763787 26213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:19.764957 26213 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:09:19.765003 26213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:19.765018 26213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:19.765030 26213 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:09:19.765043 26213 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:09:19.766435 26213 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:19.766628 26213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:19.766791 26213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:19.772508 26218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:19.772560 26221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:19.772513 26219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:19.773051 26213 server_base.cc:1034] running on GCE node
I20260925 22:09:19.773339 26213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:19.773682 26213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:19.774854 26213 hybrid_clock.cc:648] HybridClock initialized: now 1790374159774831 us; error 44 us; skew 500 ppm
Sep 25 22:09:19 dist-test-slave-7mqr krb5kdc[26200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374159, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:19.777657 26213 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 22:09:19.778134 26213 webserver.cc:460] Webserver started at http://127.0.0.1:46787/ using document root <none> and password file <none>
I20260925 22:09:19.778398 26213 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:19.778475 26213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:19.778654 26213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:19.779770 26213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "95297c77302b4616a68a680a167c4c80"
format_stamp: "Formatted at 2026-09-25 22:09:19 on dist-test-slave-7mqr"
server_key: "146501da4a09a51d5a2b42dd8387da09"
server_key_iv: "76feff1bcc8513c65d2d86eadeaa334f"
server_key_version: "encryptionkey@0"
I20260925 22:09:19.780203 26213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "95297c77302b4616a68a680a167c4c80"
format_stamp: "Formatted at 2026-09-25 22:09:19 on dist-test-slave-7mqr"
server_key: "146501da4a09a51d5a2b42dd8387da09"
server_key_iv: "76feff1bcc8513c65d2d86eadeaa334f"
server_key_version: "encryptionkey@0"
I20260925 22:09:19.781592 26213 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:09:19.782291 26228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:19.782440 26213 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:09:19.782541 26213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "95297c77302b4616a68a680a167c4c80"
format_stamp: "Formatted at 2026-09-25 22:09:19 on dist-test-slave-7mqr"
server_key: "146501da4a09a51d5a2b42dd8387da09"
server_key_iv: "76feff1bcc8513c65d2d86eadeaa334f"
server_key_version: "encryptionkey@0"
I20260925 22:09:19.782619 26213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:19.785955 26213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:19.788280 26213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:19.788432 26213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:19.792339 26213 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39889
I20260925 22:09:19.792371 26280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39889 every 8 connection(s)
I20260925 22:09:19.792685 26213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:09:19.793609 26281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:19.795755 26281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95297c77302b4616a68a680a167c4c80: Bootstrap starting.
I20260925 22:09:19.796343 26281 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95297c77302b4616a68a680a167c4c80: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:19.796638 26281 log.cc:824] T 00000000000000000000000000000000 P 95297c77302b4616a68a680a167c4c80: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:19.797287 26281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95297c77302b4616a68a680a167c4c80: No bootstrap required, opened a new log
I20260925 22:09:19.797956 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 26213
I20260925 22:09:19.798103 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:09:19.798278 5121 external_mini_cluster.cc:1428] Setting key 3e4f2bf060238f37700168f7a9adf023
I20260925 22:09:19.799466 26281 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95297c77302b4616a68a680a167c4c80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95297c77302b4616a68a680a167c4c80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39889 } }
I20260925 22:09:19.799644 26281 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95297c77302b4616a68a680a167c4c80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:19.799683 26281 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95297c77302b4616a68a680a167c4c80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95297c77302b4616a68a680a167c4c80, State: Initialized, Role: FOLLOWER
I20260925 22:09:19.799819 26281 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95297c77302b4616a68a680a167c4c80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95297c77302b4616a68a680a167c4c80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39889 } }
I20260925 22:09:19.799944 26281 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95297c77302b4616a68a680a167c4c80 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:09:19.799988 26281 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95297c77302b4616a68a680a167c4c80 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:09:19.800036 26281 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95297c77302b4616a68a680a167c4c80 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:19.800717 26285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:19.798823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.212:40533 (local address 127.0.0.1:39889)
0925 22:09:19.799026 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:19.799036 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:19.799071 (+ 35us) server_negotiation.cc:407] Connection header received
0925 22:09:19.799425 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:19.799433 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:19.799467 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:19.799618 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:19.800310 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:19.800313 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:19.800313 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:19.800317 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:19.800358 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:19.800370 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:19.800424 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:19.800525 (+ 101us) server_negotiation.cc:299] Negotiation successful
0925 22:09:19.800565 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":80,"threads_started":1}
I20260925 22:09:19.801232 26281 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95297c77302b4616a68a680a167c4c80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95297c77302b4616a68a680a167c4c80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39889 } }
I20260925 22:09:19.801342 26281 leader_election.cc:302] T 00000000000000000000000000000000 P 95297c77302b4616a68a680a167c4c80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95297c77302b4616a68a680a167c4c80; no voters:
I20260925 22:09:19.801513 26281 leader_election.cc:288] T 00000000000000000000000000000000 P 95297c77302b4616a68a680a167c4c80 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:09:19.801528 26286 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95297c77302b4616a68a680a167c4c80 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:09:19.801651 26286 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95297c77302b4616a68a680a167c4c80 [term 1 LEADER]: Becoming Leader. State: Replica: 95297c77302b4616a68a680a167c4c80, State: Running, Role: LEADER
I20260925 22:09:19.801689 5121 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:09:19.801725 26286 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95297c77302b4616a68a680a167c4c80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95297c77302b4616a68a680a167c4c80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39889 } }
I20260925 22:09:19.801769 26281 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95297c77302b4616a68a680a167c4c80 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:09:19.802058 26286 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95297c77302b4616a68a680a167c4c80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "95297c77302b4616a68a680a167c4c80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95297c77302b4616a68a680a167c4c80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39889 } } }
I20260925 22:09:19.802134 26286 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95297c77302b4616a68a680a167c4c80 [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:19.802263 26286 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95297c77302b4616a68a680a167c4c80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95297c77302b4616a68a680a167c4c80. Latest consensus state: current_term: 1 leader_uuid: "95297c77302b4616a68a680a167c4c80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95297c77302b4616a68a680a167c4c80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39889 } } }
I20260925 22:09:19.802330 26286 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95297c77302b4616a68a680a167c4c80 [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:19.802460 26292 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:09:19.803210 26292 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:09:19.804539 26285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:19.803368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.212:60607 (local address 127.0.0.1:39889)
0925 22:09:19.803554 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:19.803556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:19.803688 (+ 132us) server_negotiation.cc:407] Connection header received
0925 22:09:19.803727 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:19.803728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:19.803733 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:19.803770 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:19.804343 (+ 573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:19.804344 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:19.804344 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:19.804348 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:19.804363 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:19.804367 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:19.804434 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:19.804479 (+ 45us) server_negotiation.cc:299] Negotiation successful
0925 22:09:19.804499 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168}
I20260925 22:09:19.808454 26292 catalog_manager.cc:1347] Generated new cluster ID: 4a35bc5f14ad426a83a5670c92b73dee
I20260925 22:09:19.808503 26292 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:09:19.817634 26292 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:09:19.817812 26292 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:09:19.826308 26292 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95297c77302b4616a68a680a167c4c80: Generated new TSK 0
I20260925 22:09:19.826515 26292 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:09:19.832729 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 26213
2026-09-25T22:09:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1299 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:09:19 dist-test-slave-7mqr krb5kdc[26312](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:09:19 dist-test-slave-7mqr krb5kdc[26312](info): set up 2 sockets
Sep 25 22:09:19 dist-test-slave-7mqr krb5kdc[26312](info): commencing operation
krb5kdc: starting...
W20260925 22:09:20.872161 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:09:20 dist-test-slave-7mqr krb5kdc[26312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:09:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:09:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 22:09:20.994952 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38431
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38431
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:09:21.076356 26325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:21.076597 26325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:21.076637 26325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:21.077776 26325 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:09:21.077843 26325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:21.077867 26325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:21.077888 26325 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:09:21.077916 26325 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:09:21.079417 26325 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38431
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:21.079713 26325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:21.079979 26325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:21.084893 26330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:21.084930 26331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:21.084864 26333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:21.085067 26325 server_base.cc:1034] running on GCE node
I20260925 22:09:21.085302 26325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:21.085601 26325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:21.086768 26325 hybrid_clock.cc:648] HybridClock initialized: now 1790374161086767 us; error 41 us; skew 500 ppm
Sep 25 22:09:21 dist-test-slave-7mqr krb5kdc[26312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374161, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:21.096330 26325 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 22:09:21.096849 26325 webserver.cc:460] Webserver started at http://127.0.0.1:45437/ using document root <none> and password file <none>
I20260925 22:09:21.097100 26325 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:21.097170 26325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:21.097321 26325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:21.098434 26325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "379a9b236c344545b9600cf730768758"
format_stamp: "Formatted at 2026-09-25 22:09:21 on dist-test-slave-7mqr"
server_key: "1aa5ab83f319de790fc98116a6faa72e"
server_key_iv: "2f4630c43dbbfdea0d2409b8c59da1b6"
server_key_version: "encryptionkey@0"
I20260925 22:09:21.098798 26325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "379a9b236c344545b9600cf730768758"
format_stamp: "Formatted at 2026-09-25 22:09:21 on dist-test-slave-7mqr"
server_key: "1aa5ab83f319de790fc98116a6faa72e"
server_key_iv: "2f4630c43dbbfdea0d2409b8c59da1b6"
server_key_version: "encryptionkey@0"
I20260925 22:09:21.100274 26325 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:09:21.101148 26340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:21.101419 26325 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:09:21.101560 26325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "379a9b236c344545b9600cf730768758"
format_stamp: "Formatted at 2026-09-25 22:09:21 on dist-test-slave-7mqr"
server_key: "1aa5ab83f319de790fc98116a6faa72e"
server_key_iv: "2f4630c43dbbfdea0d2409b8c59da1b6"
server_key_version: "encryptionkey@0"
I20260925 22:09:21.101646 26325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:21.127777 26325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:21.129822 26325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:21.129997 26325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:21.134182 26325 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38431
I20260925 22:09:21.134224 26392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38431 every 8 connection(s)
I20260925 22:09:21.134622 26325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:09:21.135643 26393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:21.137899 26393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 379a9b236c344545b9600cf730768758: Bootstrap starting.
I20260925 22:09:21.138434 26393 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 379a9b236c344545b9600cf730768758: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:21.138662 26393 log.cc:824] T 00000000000000000000000000000000 P 379a9b236c344545b9600cf730768758: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:21.139284 26393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 379a9b236c344545b9600cf730768758: No bootstrap required, opened a new log
I20260925 22:09:21.140064 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 26325
I20260925 22:09:21.140184 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:09:21.140378 5121 external_mini_cluster.cc:1428] Setting key 308f81a9d933f45325e3ab3c8cd08d04
I20260925 22:09:21.141438 26393 raft_consensus.cc:348] T 00000000000000000000000000000000 P 379a9b236c344545b9600cf730768758 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "379a9b236c344545b9600cf730768758" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38431 } }
I20260925 22:09:21.141606 26393 raft_consensus.cc:374] T 00000000000000000000000000000000 P 379a9b236c344545b9600cf730768758 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:21.141644 26393 raft_consensus.cc:729] T 00000000000000000000000000000000 P 379a9b236c344545b9600cf730768758 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 379a9b236c344545b9600cf730768758, State: Initialized, Role: FOLLOWER
I20260925 22:09:21.141791 26393 consensus_queue.cc:260] T 00000000000000000000000000000000 P 379a9b236c344545b9600cf730768758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "379a9b236c344545b9600cf730768758" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38431 } }
I20260925 22:09:21.141876 26393 raft_consensus.cc:388] T 00000000000000000000000000000000 P 379a9b236c344545b9600cf730768758 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:09:21.141923 26393 raft_consensus.cc:482] T 00000000000000000000000000000000 P 379a9b236c344545b9600cf730768758 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:09:21.141975 26393 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 379a9b236c344545b9600cf730768758 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:21.142974 26393 raft_consensus.cc:504] T 00000000000000000000000000000000 P 379a9b236c344545b9600cf730768758 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "379a9b236c344545b9600cf730768758" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38431 } }
I20260925 22:09:21.143061 26393 leader_election.cc:302] T 00000000000000000000000000000000 P 379a9b236c344545b9600cf730768758 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 379a9b236c344545b9600cf730768758; no voters:
I20260925 22:09:21.143241 26393 leader_election.cc:288] T 00000000000000000000000000000000 P 379a9b236c344545b9600cf730768758 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:09:21.143285 26398 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 379a9b236c344545b9600cf730768758 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:09:21.143402 26398 raft_consensus.cc:686] T 00000000000000000000000000000000 P 379a9b236c344545b9600cf730768758 [term 1 LEADER]: Becoming Leader. State: Replica: 379a9b236c344545b9600cf730768758, State: Running, Role: LEADER
I20260925 22:09:21.143501 26393 sys_catalog.cc:564] T 00000000000000000000000000000000 P 379a9b236c344545b9600cf730768758 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:09:21.143483 26398 consensus_queue.cc:237] T 00000000000000000000000000000000 P 379a9b236c344545b9600cf730768758 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "379a9b236c344545b9600cf730768758" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38431 } }
I20260925 22:09:21.143848 26398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 379a9b236c344545b9600cf730768758 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 379a9b236c344545b9600cf730768758. Latest consensus state: current_term: 1 leader_uuid: "379a9b236c344545b9600cf730768758" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "379a9b236c344545b9600cf730768758" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38431 } } }
I20260925 22:09:21.143954 26398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 379a9b236c344545b9600cf730768758 [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:21.144243 26402 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:09:21.144716 26402 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:09:21.145534 26399 sys_catalog.cc:455] T 00000000000000000000000000000000 P 379a9b236c344545b9600cf730768758 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "379a9b236c344545b9600cf730768758" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "379a9b236c344545b9600cf730768758" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38431 } } }
I20260925 22:09:21.145610 26399 sys_catalog.cc:458] T 00000000000000000000000000000000 P 379a9b236c344545b9600cf730768758 [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:21.146219 26402 catalog_manager.cc:1347] Generated new cluster ID: 99ea0928a3eb44bbbf854dbc1446e8f3
I20260925 22:09:21.146270 26402 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:09:21 dist-test-slave-7mqr krb5kdc[26312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 22:09:21.152453 26397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:21.140943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.212:35639 (local address 127.0.0.1:38431)
0925 22:09:21.141137 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:21.141146 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:21.141171 (+ 25us) server_negotiation.cc:407] Connection header received
0925 22:09:21.141416 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:21.141422 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:21.141471 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:21.141588 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:21.142457 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:21.143639 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:21.144979 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:21.145642 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:21.145752 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:21.148714 (+ 2962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:21.148719 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:21.148719 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:21.148731 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:21.150729 (+ 1998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:21.151172 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:21.151175 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:21.151175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:21.151296 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:21.151552 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:21.151553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:21.151553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:21.151840 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:21.152010 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:21.152167 (+ 157us) server_negotiation.cc:299] Negotiation successful
0925 22:09:21.152266 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":90,"threads_started":1}
I20260925 22:09:21.153440 5121 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:09:21.161772 26397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:21.154757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.212:40745 (local address 127.0.0.1:38431)
0925 22:09:21.154792 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:21.154793 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:21.154822 (+ 29us) server_negotiation.cc:407] Connection header received
0925 22:09:21.154875 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:21.154875 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:21.154884 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:21.154950 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:21.155516 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:21.156309 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:21.157237 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:21.157561 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:21.157634 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:21.158923 (+ 1289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:21.158925 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:21.158925 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:21.158932 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:21.160536 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:21.160941 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:21.160942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:21.160942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:21.160998 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:21.161230 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:21.161230 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:21.161231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:21.161374 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:21.161477 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:21.161634 (+ 157us) server_negotiation.cc:299] Negotiation successful
0925 22:09:21.161698 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":18}
I20260925 22:09:21.162160 26402 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:09:21.162698 26402 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:09:21.169850 26402 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 379a9b236c344545b9600cf730768758: Generated new TSK 0
I20260925 22:09:21.170032 26402 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:09:21.176740 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 26325
W20260925 22:09:21.180825 26310 connection.cc:535] client connection to 127.0.0.1:38431 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38431): Connection reset by peer (error 104)
2026-09-25T22:09:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1343 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:09:21 dist-test-slave-7mqr krb5kdc[26424](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:09:21 dist-test-slave-7mqr krb5kdc[26424](info): set up 2 sockets
Sep 25 22:09:21 dist-test-slave-7mqr krb5kdc[26424](info): commencing operation
krb5kdc: starting...
W20260925 22:09:22.232048 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:09:22 dist-test-slave-7mqr krb5kdc[26424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:09:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:09:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 22:09:22.350373 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35353
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35353
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:09:22.429811 26437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:22.430047 26437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:22.430085 26437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:22.431212 26437 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:09:22.431281 26437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:22.431304 26437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:22.431326 26437 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:09:22.431355 26437 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:09:22.432845 26437 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35353
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:22.433126 26437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:22.433351 26437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:22.437989 26443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:22.438081 26437 server_base.cc:1034] running on GCE node
W20260925 22:09:22.438009 26442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:22.437981 26445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:22.438455 26437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:22.438755 26437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:22.439977 26437 hybrid_clock.cc:648] HybridClock initialized: now 1790374162439940 us; error 25 us; skew 500 ppm
Sep 25 22:09:22 dist-test-slave-7mqr krb5kdc[26424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374162, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:22.442261 26437 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 22:09:22.442698 26437 webserver.cc:460] Webserver started at http://127.0.0.1:45745/ using document root <none> and password file <none>
I20260925 22:09:22.442879 26437 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:22.442919 26437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:22.443010 26437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:22.444036 26437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "787780d17cb648c6aeb303bf8d41b8c2"
format_stamp: "Formatted at 2026-09-25 22:09:22 on dist-test-slave-7mqr"
server_key: "a3dce343895579c713fc5c137637eda7"
server_key_iv: "0f1727176d609ca12526e066bd50290a"
server_key_version: "encryptionkey@0"
I20260925 22:09:22.444327 26437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "787780d17cb648c6aeb303bf8d41b8c2"
format_stamp: "Formatted at 2026-09-25 22:09:22 on dist-test-slave-7mqr"
server_key: "a3dce343895579c713fc5c137637eda7"
server_key_iv: "0f1727176d609ca12526e066bd50290a"
server_key_version: "encryptionkey@0"
I20260925 22:09:22.445544 26437 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:09:22.446264 26452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:22.446482 26437 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:09:22.446573 26437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "787780d17cb648c6aeb303bf8d41b8c2"
format_stamp: "Formatted at 2026-09-25 22:09:22 on dist-test-slave-7mqr"
server_key: "a3dce343895579c713fc5c137637eda7"
server_key_iv: "0f1727176d609ca12526e066bd50290a"
server_key_version: "encryptionkey@0"
I20260925 22:09:22.446630 26437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:22.461181 26437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:22.462981 26437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:22.463099 26437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:22.466923 26437 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35353
I20260925 22:09:22.467000 26504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35353 every 8 connection(s)
I20260925 22:09:22.467329 26437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:09:22.468223 26505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:22.470427 26505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 787780d17cb648c6aeb303bf8d41b8c2: Bootstrap starting.
I20260925 22:09:22.471016 26505 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 787780d17cb648c6aeb303bf8d41b8c2: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:22.471294 26505 log.cc:824] T 00000000000000000000000000000000 P 787780d17cb648c6aeb303bf8d41b8c2: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:22.472069 26505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 787780d17cb648c6aeb303bf8d41b8c2: No bootstrap required, opened a new log
I20260925 22:09:22.473709 26505 raft_consensus.cc:348] T 00000000000000000000000000000000 P 787780d17cb648c6aeb303bf8d41b8c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787780d17cb648c6aeb303bf8d41b8c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35353 } }
I20260925 22:09:22.473868 26505 raft_consensus.cc:374] T 00000000000000000000000000000000 P 787780d17cb648c6aeb303bf8d41b8c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:22.473914 26505 raft_consensus.cc:729] T 00000000000000000000000000000000 P 787780d17cb648c6aeb303bf8d41b8c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787780d17cb648c6aeb303bf8d41b8c2, State: Initialized, Role: FOLLOWER
I20260925 22:09:22.474086 26505 consensus_queue.cc:260] T 00000000000000000000000000000000 P 787780d17cb648c6aeb303bf8d41b8c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787780d17cb648c6aeb303bf8d41b8c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35353 } }
I20260925 22:09:22.474170 26505 raft_consensus.cc:388] T 00000000000000000000000000000000 P 787780d17cb648c6aeb303bf8d41b8c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:09:22.474213 26505 raft_consensus.cc:482] T 00000000000000000000000000000000 P 787780d17cb648c6aeb303bf8d41b8c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:09:22.474264 26505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 787780d17cb648c6aeb303bf8d41b8c2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:22.474772 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 26437
I20260925 22:09:22.474786 26505 raft_consensus.cc:504] T 00000000000000000000000000000000 P 787780d17cb648c6aeb303bf8d41b8c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787780d17cb648c6aeb303bf8d41b8c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35353 } }
I20260925 22:09:22.474864 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:09:22.474902 26505 leader_election.cc:302] T 00000000000000000000000000000000 P 787780d17cb648c6aeb303bf8d41b8c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 787780d17cb648c6aeb303bf8d41b8c2; no voters:
I20260925 22:09:22.475096 5121 external_mini_cluster.cc:1428] Setting key 89f6c969a37f53ed39d676395c1dc78d
I20260925 22:09:22.475118 26505 leader_election.cc:288] T 00000000000000000000000000000000 P 787780d17cb648c6aeb303bf8d41b8c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:09:22.475142 26508 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 787780d17cb648c6aeb303bf8d41b8c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:09:22.475330 26508 raft_consensus.cc:686] T 00000000000000000000000000000000 P 787780d17cb648c6aeb303bf8d41b8c2 [term 1 LEADER]: Becoming Leader. State: Replica: 787780d17cb648c6aeb303bf8d41b8c2, State: Running, Role: LEADER
I20260925 22:09:22.475420 26505 sys_catalog.cc:564] T 00000000000000000000000000000000 P 787780d17cb648c6aeb303bf8d41b8c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:09:22.475461 26508 consensus_queue.cc:237] T 00000000000000000000000000000000 P 787780d17cb648c6aeb303bf8d41b8c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787780d17cb648c6aeb303bf8d41b8c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35353 } }
I20260925 22:09:22.476511 26510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 787780d17cb648c6aeb303bf8d41b8c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 787780d17cb648c6aeb303bf8d41b8c2. Latest consensus state: current_term: 1 leader_uuid: "787780d17cb648c6aeb303bf8d41b8c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787780d17cb648c6aeb303bf8d41b8c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35353 } } }
I20260925 22:09:22.476606 26510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 787780d17cb648c6aeb303bf8d41b8c2 [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:22.476874 26509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 787780d17cb648c6aeb303bf8d41b8c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "787780d17cb648c6aeb303bf8d41b8c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787780d17cb648c6aeb303bf8d41b8c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35353 } } }
I20260925 22:09:22.476869 26523 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:09:22.477620 26509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 787780d17cb648c6aeb303bf8d41b8c2 [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:22.477974 26523 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:09:22.479689 26523 catalog_manager.cc:1347] Generated new cluster ID: 757f71073a0a4232a94e71711fd9dc1f
I20260925 22:09:22.479763 26523 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:09:22.481640 26512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:22.475587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.212:33737 (local address 127.0.0.1:35353)
0925 22:09:22.476074 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:22.476081 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:22.476128 (+ 47us) server_negotiation.cc:407] Connection header received
0925 22:09:22.476249 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:22.476254 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:22.476298 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:22.476435 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:22.476713 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:22.478156 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:22.479110 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:22.479718 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:22.479830 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:22.480827 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:22.480832 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:22.480832 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:22.481066 (+ 234us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:22.481105 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:22.481120 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:22.481211 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:22.481344 (+ 133us) server_negotiation.cc:299] Negotiation successful
0925 22:09:22.481375 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":87,"threads_started":1}
I20260925 22:09:22.482697 5121 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:09:22.487879 26512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:22.484300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.212:51837 (local address 127.0.0.1:35353)
0925 22:09:22.484332 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:22.484333 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:22.484341 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:09:22.484413 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:22.484414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:22.484424 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:22.484477 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:22.484633 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:22.485402 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:22.486490 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:22.486840 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:22.486914 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:22.487629 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:22.487630 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:22.487630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:22.487634 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:22.487650 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:22.487655 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:22.487718 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:22.487796 (+ 78us) server_negotiation.cc:299] Negotiation successful
0925 22:09:22.487809 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 22:09:22.496810 26523 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:09:22.497520 26523 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:09:22.502939 26523 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 787780d17cb648c6aeb303bf8d41b8c2: Generated new TSK 0
I20260925 22:09:22.503113 26523 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:09:22.509572 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 26437
W20260925 22:09:22.513800 26422 connection.cc:535] client connection to 127.0.0.1:35353 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35353): Connection reset by peer (error 104)
2026-09-25T22:09:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1332 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:09:22 dist-test-slave-7mqr krb5kdc[26536](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:09:22 dist-test-slave-7mqr krb5kdc[26536](info): set up 2 sockets
Sep 25 22:09:22 dist-test-slave-7mqr krb5kdc[26536](info): commencing operation
krb5kdc: starting...
W20260925 22:09:23.540107 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:09:23 dist-test-slave-7mqr krb5kdc[26536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:09:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:09:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 22:09:23.660162 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37103
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37103
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:09:23.746955 26549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:23.747174 26549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:23.747197 26549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:23.748360 26549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:09:23.748402 26549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:23.748416 26549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:23.748430 26549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:09:23.748443 26549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:09:23.749851 26549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37103
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373908987421-5121-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:23.750052 26549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:23.750221 26549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:23.754904 26557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:23.754927 26555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:23.755016 26554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:23.755110 26549 server_base.cc:1034] running on GCE node
I20260925 22:09:23.755540 26549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:23.756052 26549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:23.757249 26549 hybrid_clock.cc:648] HybridClock initialized: now 1790374163757246 us; error 43 us; skew 500 ppm
Sep 25 22:09:23 dist-test-slave-7mqr krb5kdc[26536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374163, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:23.759689 26549 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 22:09:23.760210 26549 webserver.cc:460] Webserver started at http://127.0.0.1:40963/ using document root <none> and password file <none>
I20260925 22:09:23.760448 26549 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:23.760517 26549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:23.760670 26549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:23.761775 26549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "e9a50c473a234bf7bb68adb55a67c993"
format_stamp: "Formatted at 2026-09-25 22:09:23 on dist-test-slave-7mqr"
server_key: "457623c85aa31f36b73b2d8ecb6d505e"
server_key_iv: "7fbaf5773ab55449d513b33e91e68db5"
server_key_version: "encryptionkey@0"
I20260925 22:09:23.762147 26549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "e9a50c473a234bf7bb68adb55a67c993"
format_stamp: "Formatted at 2026-09-25 22:09:23 on dist-test-slave-7mqr"
server_key: "457623c85aa31f36b73b2d8ecb6d505e"
server_key_iv: "7fbaf5773ab55449d513b33e91e68db5"
server_key_version: "encryptionkey@0"
I20260925 22:09:23.763564 26549 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:09:23.764415 26564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:23.764654 26549 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 22:09:23.764755 26549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "e9a50c473a234bf7bb68adb55a67c993"
format_stamp: "Formatted at 2026-09-25 22:09:23 on dist-test-slave-7mqr"
server_key: "457623c85aa31f36b73b2d8ecb6d505e"
server_key_iv: "7fbaf5773ab55449d513b33e91e68db5"
server_key_version: "encryptionkey@0"
I20260925 22:09:23.764835 26549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:23.768189 26549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:23.770539 26549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:23.770699 26549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:23.774490 26549 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37103
I20260925 22:09:23.774551 26616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37103 every 8 connection(s)
I20260925 22:09:23.774924 26549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:09:23.775979 26617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:23.778213 26617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9a50c473a234bf7bb68adb55a67c993: Bootstrap starting.
I20260925 22:09:23.778795 26617 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9a50c473a234bf7bb68adb55a67c993: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:23.779081 26617 log.cc:824] T 00000000000000000000000000000000 P e9a50c473a234bf7bb68adb55a67c993: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:23.779762 26617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9a50c473a234bf7bb68adb55a67c993: No bootstrap required, opened a new log
I20260925 22:09:23.781333 26617 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9a50c473a234bf7bb68adb55a67c993 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9a50c473a234bf7bb68adb55a67c993" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37103 } }
I20260925 22:09:23.781476 26617 raft_consensus.cc:374] T 00000000000000000000000000000000 P e9a50c473a234bf7bb68adb55a67c993 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:23.781531 26617 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9a50c473a234bf7bb68adb55a67c993 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a50c473a234bf7bb68adb55a67c993, State: Initialized, Role: FOLLOWER
I20260925 22:09:23.781697 26617 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9a50c473a234bf7bb68adb55a67c993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9a50c473a234bf7bb68adb55a67c993" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37103 } }
I20260925 22:09:23.781761 26617 raft_consensus.cc:388] T 00000000000000000000000000000000 P e9a50c473a234bf7bb68adb55a67c993 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:09:23.781782 26617 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9a50c473a234bf7bb68adb55a67c993 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:09:23.781845 26617 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9a50c473a234bf7bb68adb55a67c993 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:23.782382 26617 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9a50c473a234bf7bb68adb55a67c993 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9a50c473a234bf7bb68adb55a67c993" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37103 } }
I20260925 22:09:23.782496 26617 leader_election.cc:302] T 00000000000000000000000000000000 P e9a50c473a234bf7bb68adb55a67c993 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9a50c473a234bf7bb68adb55a67c993; no voters:
I20260925 22:09:23.782675 26617 leader_election.cc:288] T 00000000000000000000000000000000 P e9a50c473a234bf7bb68adb55a67c993 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:09:23.782701 26620 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9a50c473a234bf7bb68adb55a67c993 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:09:23.782902 26620 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9a50c473a234bf7bb68adb55a67c993 [term 1 LEADER]: Becoming Leader. State: Replica: e9a50c473a234bf7bb68adb55a67c993, State: Running, Role: LEADER
I20260925 22:09:23.783005 26617 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9a50c473a234bf7bb68adb55a67c993 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:09:23.783035 26620 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9a50c473a234bf7bb68adb55a67c993 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9a50c473a234bf7bb68adb55a67c993" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37103 } }
I20260925 22:09:23.783427 26622 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9a50c473a234bf7bb68adb55a67c993 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9a50c473a234bf7bb68adb55a67c993. Latest consensus state: current_term: 1 leader_uuid: "e9a50c473a234bf7bb68adb55a67c993" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9a50c473a234bf7bb68adb55a67c993" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37103 } } }
I20260925 22:09:23.783562 26621 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9a50c473a234bf7bb68adb55a67c993 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e9a50c473a234bf7bb68adb55a67c993" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9a50c473a234bf7bb68adb55a67c993" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37103 } } }
I20260925 22:09:23.783598 26622 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9a50c473a234bf7bb68adb55a67c993 [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:23.783661 26621 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9a50c473a234bf7bb68adb55a67c993 [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:23.784166 26629 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:09:23.784751 26629 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:09:23.784999 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 26549
I20260925 22:09:23.785148 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:09:23.785441 5121 external_mini_cluster.cc:1428] Setting key 6f5c09e27089351c9d1107a4e1477a74
I20260925 22:09:23.788417 26629 catalog_manager.cc:1347] Generated new cluster ID: 01b04ebc801142b091aa466f61c49646
I20260925 22:09:23.788475 26629 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:09:23.788626 26640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:23.786549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.212:58773 (local address 127.0.0.1:37103)
0925 22:09:23.786742 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:23.786752 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:23.786783 (+ 31us) server_negotiation.cc:407] Connection header received
0925 22:09:23.786897 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:23.786904 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:23.786936 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:23.787079 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:23.788016 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:23.788019 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:23.788020 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:23.788024 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:23.788071 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:23.788087 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:23.788359 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:23.788393 (+ 34us) server_negotiation.cc:299] Negotiation successful
0925 22:09:23.788445 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":75,"threads_started":1}
I20260925 22:09:23.792904 5121 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:09:23.793481 26629 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:09:23.793622 26629 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:09:23.795609 26640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:23.794566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.212:36717 (local address 127.0.0.1:37103)
0925 22:09:23.794604 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:23.794606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:23.794667 (+ 61us) server_negotiation.cc:407] Connection header received
0925 22:09:23.794731 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:23.794732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:23.794737 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:23.794791 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:23.795397 (+ 606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:23.795398 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:23.795398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:23.795402 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:23.795416 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:23.795421 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:23.795476 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:23.795536 (+ 60us) server_negotiation.cc:299] Negotiation successful
0925 22:09:23.795550 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 22:09:23.800314 26629 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e9a50c473a234bf7bb68adb55a67c993: Generated new TSK 0
I20260925 22:09:23.800513 26629 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:09:23.802721 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 26549
2026-09-25T22:09:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1293 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:09:23 dist-test-slave-7mqr krb5kdc[26648](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:09:23 dist-test-slave-7mqr krb5kdc[26648](info): set up 2 sockets
Sep 25 22:09:23 dist-test-slave-7mqr krb5kdc[26648](info): commencing operation
krb5kdc: starting...
W20260925 22:09:24.848073 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:09:24 dist-test-slave-7mqr krb5kdc[26648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:09:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:09:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.0.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
I20260925 22:09:24.967430 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:35483
--webserver_interface=127.5.0.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:33997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:35483
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:09:25.047158 26661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:25.047410 26661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:25.047446 26661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:25.048647 26661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:09:25.048723 26661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:25.048748 26661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:25.048768 26661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:09:25.048799 26661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:09:25.050285 26661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:33997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:35483
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:35483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:25.050568 26661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:25.050810 26661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:25.055632 26669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:25.055624 26666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:25.055634 26667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:25.055794 26661 server_base.cc:1034] running on GCE node
I20260925 22:09:25.056145 26661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:25.056402 26661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:25.057561 26661 hybrid_clock.cc:648] HybridClock initialized: now 1790374165057539 us; error 40 us; skew 500 ppm
Sep 25 22:09:25 dist-test-slave-7mqr krb5kdc[26648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374165, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:25.060161 26661 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:09:25.060648 26661 webserver.cc:460] Webserver started at http://127.5.0.126:36187/ using document root <none> and password file <none>
I20260925 22:09:25.060900 26661 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:25.060971 26661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:25.061125 26661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:25.062243 26661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "4159c53d4c7e4b6bb339d46de0312398"
format_stamp: "Formatted at 2026-09-25 22:09:25 on dist-test-slave-7mqr"
server_key: "2680f5c846614b23c9e9e97df4acb7f8"
server_key_iv: "4d6597647fd0226c63e44ef16b7731ef"
server_key_version: "encryptionkey@0"
I20260925 22:09:25.062614 26661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "4159c53d4c7e4b6bb339d46de0312398"
format_stamp: "Formatted at 2026-09-25 22:09:25 on dist-test-slave-7mqr"
server_key: "2680f5c846614b23c9e9e97df4acb7f8"
server_key_iv: "4d6597647fd0226c63e44ef16b7731ef"
server_key_version: "encryptionkey@0"
I20260925 22:09:25.064021 26661 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 22:09:25.064782 26676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:25.064950 26661 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:09:25.065066 26661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "4159c53d4c7e4b6bb339d46de0312398"
format_stamp: "Formatted at 2026-09-25 22:09:25 on dist-test-slave-7mqr"
server_key: "2680f5c846614b23c9e9e97df4acb7f8"
server_key_iv: "4d6597647fd0226c63e44ef16b7731ef"
server_key_version: "encryptionkey@0"
I20260925 22:09:25.065145 26661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:25.081972 26661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:25.083830 26661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:25.084033 26661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:25.087903 26661 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:35483
I20260925 22:09:25.088009 26728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:35483 every 8 connection(s)
I20260925 22:09:25.088336 26661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:09:25.089177 26729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:25.091634 26729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4159c53d4c7e4b6bb339d46de0312398: Bootstrap starting.
I20260925 22:09:25.092145 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 26661
I20260925 22:09:25.092268 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:09:25.092401 26729 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4159c53d4c7e4b6bb339d46de0312398: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:25.092516 5121 external_mini_cluster.cc:1428] Setting key 0caadfe26c4b6109e3c3c357de869dd2
I20260925 22:09:25.092725 26729 log.cc:824] T 00000000000000000000000000000000 P 4159c53d4c7e4b6bb339d46de0312398: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:25.093459 26729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4159c53d4c7e4b6bb339d46de0312398: No bootstrap required, opened a new log
I20260925 22:09:25.095736 26729 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4159c53d4c7e4b6bb339d46de0312398 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4159c53d4c7e4b6bb339d46de0312398" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 35483 } }
I20260925 22:09:25.095940 26729 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4159c53d4c7e4b6bb339d46de0312398 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:25.095988 26729 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4159c53d4c7e4b6bb339d46de0312398 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4159c53d4c7e4b6bb339d46de0312398, State: Initialized, Role: FOLLOWER
I20260925 22:09:25.096112 26729 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4159c53d4c7e4b6bb339d46de0312398 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4159c53d4c7e4b6bb339d46de0312398" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 35483 } }
I20260925 22:09:25.096189 26729 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4159c53d4c7e4b6bb339d46de0312398 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:09:25.096218 26729 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4159c53d4c7e4b6bb339d46de0312398 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:09:25.096262 26729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4159c53d4c7e4b6bb339d46de0312398 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:25.096997 26729 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4159c53d4c7e4b6bb339d46de0312398 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4159c53d4c7e4b6bb339d46de0312398" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 35483 } }
I20260925 22:09:25.097141 26729 leader_election.cc:302] T 00000000000000000000000000000000 P 4159c53d4c7e4b6bb339d46de0312398 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4159c53d4c7e4b6bb339d46de0312398; no voters:
I20260925 22:09:25.097318 26729 leader_election.cc:288] T 00000000000000000000000000000000 P 4159c53d4c7e4b6bb339d46de0312398 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:09:25.097345 26734 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4159c53d4c7e4b6bb339d46de0312398 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:09:25.097532 26734 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4159c53d4c7e4b6bb339d46de0312398 [term 1 LEADER]: Becoming Leader. State: Replica: 4159c53d4c7e4b6bb339d46de0312398, State: Running, Role: LEADER
I20260925 22:09:25.097630 26729 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4159c53d4c7e4b6bb339d46de0312398 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:09:25.097671 26734 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4159c53d4c7e4b6bb339d46de0312398 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4159c53d4c7e4b6bb339d46de0312398" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 35483 } }
I20260925 22:09:25.098133 26736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4159c53d4c7e4b6bb339d46de0312398 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4159c53d4c7e4b6bb339d46de0312398. Latest consensus state: current_term: 1 leader_uuid: "4159c53d4c7e4b6bb339d46de0312398" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4159c53d4c7e4b6bb339d46de0312398" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 35483 } } }
I20260925 22:09:25.098243 26736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4159c53d4c7e4b6bb339d46de0312398 [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:25.098436 26742 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:09:25.098949 26742 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 25 22:09:25 dist-test-slave-7mqr krb5kdc[26648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:09:25.099845 26735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4159c53d4c7e4b6bb339d46de0312398 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4159c53d4c7e4b6bb339d46de0312398" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4159c53d4c7e4b6bb339d46de0312398" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 35483 } } }
I20260925 22:09:25.100111 26735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4159c53d4c7e4b6bb339d46de0312398 [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:25.100991 26742 catalog_manager.cc:1347] Generated new cluster ID: 06b0a76b1f994f61b0d8db759019da87
I20260925 22:09:25.101043 26742 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:09:25.103514 26733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:25.093007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41637 (local address 127.5.0.126:35483)
0925 22:09:25.093179 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:25.093190 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:25.093231 (+ 41us) server_negotiation.cc:407] Connection header received
0925 22:09:25.093721 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:25.093728 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:25.093774 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:25.093888 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:25.094712 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:25.095559 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:25.096725 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:25.097146 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:25.097259 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:25.100186 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:25.100190 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:25.100191 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:25.100203 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:25.101999 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:25.102386 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:25.102389 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:25.102389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:25.102479 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:25.102714 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:25.102715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:25.102715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:25.102941 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:25.103120 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:25.103213 (+ 93us) server_negotiation.cc:299] Negotiation successful
0925 22:09:25.103309 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":74,"threads_started":1}
I20260925 22:09:25.104441 5121 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:09:25.110756 26742 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:09:25.111443 26742 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:09:25.112001 26733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:25.105724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58985 (local address 127.5.0.126:35483)
0925 22:09:25.105760 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:25.105762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:25.105882 (+ 120us) server_negotiation.cc:407] Connection header received
0925 22:09:25.105957 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:25.105958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:25.105966 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:25.106010 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:25.106505 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:25.106995 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:25.107689 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:25.107969 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:25.108057 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:25.109196 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:25.109197 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:25.109198 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:25.109205 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:25.110742 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:25.111124 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:25.111125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:25.111126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:25.111176 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:25.111403 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:25.111404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:25.111405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:25.111534 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:25.111630 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:25.111792 (+ 162us) server_negotiation.cc:299] Negotiation successful
0925 22:09:25.111853 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 22:09:25.117462 26742 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4159c53d4c7e4b6bb339d46de0312398: Generated new TSK 0
I20260925 22:09:25.117652 26742 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:09:25.122980 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 26661
W20260925 22:09:25.127233 26646 connection.cc:535] client connection to 127.5.0.126:35483 recv error: Network error: failed to read from TLS socket (remote: 127.5.0.126:35483): Connection reset by peer (error 104)
2026-09-25T22:09:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1320 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:09:25 dist-test-slave-7mqr krb5kdc[26760](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:09:25 dist-test-slave-7mqr krb5kdc[26760](info): set up 2 sockets
Sep 25 22:09:25 dist-test-slave-7mqr krb5kdc[26760](info): commencing operation
krb5kdc: starting...
W20260925 22:09:26.155905 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:09:26 dist-test-slave-7mqr krb5kdc[26760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374166, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:09:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:09:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.0.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
I20260925 22:09:26.274998 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:37309
--webserver_interface=127.5.0.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:44335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:37309
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:09:26.354750 26773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:26.354947 26773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:26.354975 26773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:26.356067 26773 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:09:26.356103 26773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:26.356117 26773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:26.356130 26773 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:09:26.356142 26773 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:09:26.357520 26773 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:44335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:37309
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:37309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:26.357707 26773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:26.357869 26773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:26.362534 26778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:26.362632 26773 server_base.cc:1034] running on GCE node
W20260925 22:09:26.362530 26781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:26.362565 26779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:26.363000 26773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:26.363236 26773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:26.364449 26773 hybrid_clock.cc:648] HybridClock initialized: now 1790374166364437 us; error 38 us; skew 500 ppm
Sep 25 22:09:26 dist-test-slave-7mqr krb5kdc[26760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374166, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:26.370997 26773 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:09:26.371419 26773 webserver.cc:460] Webserver started at http://127.5.0.126:43581/ using document root <none> and password file <none>
I20260925 22:09:26.371655 26773 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:26.371698 26773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:26.371865 26773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:26.372944 26773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "e62a2f376c0e46578af7df0333871c56"
format_stamp: "Formatted at 2026-09-25 22:09:26 on dist-test-slave-7mqr"
server_key: "ae9521e82a7d5ed1f4747c614238343d"
server_key_iv: "09c179d4b19dac9f0e11736b909583ed"
server_key_version: "encryptionkey@0"
I20260925 22:09:26.373287 26773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "e62a2f376c0e46578af7df0333871c56"
format_stamp: "Formatted at 2026-09-25 22:09:26 on dist-test-slave-7mqr"
server_key: "ae9521e82a7d5ed1f4747c614238343d"
server_key_iv: "09c179d4b19dac9f0e11736b909583ed"
server_key_version: "encryptionkey@0"
I20260925 22:09:26.374557 26773 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 22:09:26.375285 26788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:26.375483 26773 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:09:26.375555 26773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "e62a2f376c0e46578af7df0333871c56"
format_stamp: "Formatted at 2026-09-25 22:09:26 on dist-test-slave-7mqr"
server_key: "ae9521e82a7d5ed1f4747c614238343d"
server_key_iv: "09c179d4b19dac9f0e11736b909583ed"
server_key_version: "encryptionkey@0"
I20260925 22:09:26.375645 26773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:26.385934 26773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:26.387776 26773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:26.388010 26773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:26.391793 26773 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:37309
I20260925 22:09:26.391839 26840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:37309 every 8 connection(s)
I20260925 22:09:26.392216 26773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:09:26.393007 26841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:26.395177 26841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e62a2f376c0e46578af7df0333871c56: Bootstrap starting.
I20260925 22:09:26.395746 26841 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e62a2f376c0e46578af7df0333871c56: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:26.396077 26841 log.cc:824] T 00000000000000000000000000000000 P e62a2f376c0e46578af7df0333871c56: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:26.396785 26841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e62a2f376c0e46578af7df0333871c56: No bootstrap required, opened a new log
I20260925 22:09:26.398316 26841 raft_consensus.cc:348] T 00000000000000000000000000000000 P e62a2f376c0e46578af7df0333871c56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e62a2f376c0e46578af7df0333871c56" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 37309 } }
I20260925 22:09:26.398449 26841 raft_consensus.cc:374] T 00000000000000000000000000000000 P e62a2f376c0e46578af7df0333871c56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:26.398527 26841 raft_consensus.cc:729] T 00000000000000000000000000000000 P e62a2f376c0e46578af7df0333871c56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e62a2f376c0e46578af7df0333871c56, State: Initialized, Role: FOLLOWER
I20260925 22:09:26.398710 26841 consensus_queue.cc:260] T 00000000000000000000000000000000 P e62a2f376c0e46578af7df0333871c56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e62a2f376c0e46578af7df0333871c56" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 37309 } }
I20260925 22:09:26.398792 26841 raft_consensus.cc:388] T 00000000000000000000000000000000 P e62a2f376c0e46578af7df0333871c56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:09:26.398815 26841 raft_consensus.cc:482] T 00000000000000000000000000000000 P e62a2f376c0e46578af7df0333871c56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:09:26.398842 26841 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e62a2f376c0e46578af7df0333871c56 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:26.399659 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 26773
I20260925 22:09:26.399664 26841 raft_consensus.cc:504] T 00000000000000000000000000000000 P e62a2f376c0e46578af7df0333871c56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e62a2f376c0e46578af7df0333871c56" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 37309 } }
I20260925 22:09:26.399801 26841 leader_election.cc:302] T 00000000000000000000000000000000 P e62a2f376c0e46578af7df0333871c56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e62a2f376c0e46578af7df0333871c56; no voters:
I20260925 22:09:26.399818 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:09:26.399994 26841 leader_election.cc:288] T 00000000000000000000000000000000 P e62a2f376c0e46578af7df0333871c56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:09:26.400029 26844 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e62a2f376c0e46578af7df0333871c56 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:09:26.400125 5121 external_mini_cluster.cc:1428] Setting key 84bf0bc2005774fbde5e564b68121e17
I20260925 22:09:26.400154 26844 raft_consensus.cc:686] T 00000000000000000000000000000000 P e62a2f376c0e46578af7df0333871c56 [term 1 LEADER]: Becoming Leader. State: Replica: e62a2f376c0e46578af7df0333871c56, State: Running, Role: LEADER
I20260925 22:09:26.400224 26844 consensus_queue.cc:237] T 00000000000000000000000000000000 P e62a2f376c0e46578af7df0333871c56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e62a2f376c0e46578af7df0333871c56" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 37309 } }
I20260925 22:09:26.400257 26841 sys_catalog.cc:564] T 00000000000000000000000000000000 P e62a2f376c0e46578af7df0333871c56 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:09:26.400607 26845 sys_catalog.cc:455] T 00000000000000000000000000000000 P e62a2f376c0e46578af7df0333871c56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e62a2f376c0e46578af7df0333871c56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e62a2f376c0e46578af7df0333871c56" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 37309 } } }
I20260925 22:09:26.400713 26845 sys_catalog.cc:458] T 00000000000000000000000000000000 P e62a2f376c0e46578af7df0333871c56 [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:26.401043 26846 sys_catalog.cc:455] T 00000000000000000000000000000000 P e62a2f376c0e46578af7df0333871c56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e62a2f376c0e46578af7df0333871c56. Latest consensus state: current_term: 1 leader_uuid: "e62a2f376c0e46578af7df0333871c56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e62a2f376c0e46578af7df0333871c56" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 37309 } } }
I20260925 22:09:26.401134 26846 sys_catalog.cc:458] T 00000000000000000000000000000000 P e62a2f376c0e46578af7df0333871c56 [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:26.401036 26853 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:09:26.402005 26853 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:09:26.403954 26853 catalog_manager.cc:1347] Generated new cluster ID: 6db0cbb04bf641528218f09f219fb25c
I20260925 22:09:26.404011 26853 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:09:26.406899 26850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:26.400658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35243 (local address 127.5.0.126:37309)
0925 22:09:26.400832 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:26.400843 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:26.400880 (+ 37us) server_negotiation.cc:407] Connection header received
0925 22:09:26.401055 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:26.401063 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:26.401119 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:26.401592 (+ 473us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:26.401724 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:26.403313 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:26.404662 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:26.405294 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:26.405416 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:26.406189 (+ 773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:26.406193 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:26.406193 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:26.406428 (+ 235us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:26.406466 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:26.406481 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:26.406573 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:26.406664 (+ 91us) server_negotiation.cc:299] Negotiation successful
0925 22:09:26.406694 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":66,"threads_started":1}
I20260925 22:09:26.407949 5121 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:09:26.415797 26850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:26.412194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39377 (local address 127.5.0.126:37309)
0925 22:09:26.412229 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:26.412230 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:26.412367 (+ 137us) server_negotiation.cc:407] Connection header received
0925 22:09:26.412429 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:26.412430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:26.412441 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:26.412493 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:26.412645 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:26.413410 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:26.414452 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:26.414781 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:26.414852 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:26.415549 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:26.415550 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:26.415551 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:26.415555 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:26.415569 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:26.415573 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:26.415634 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:26.415709 (+ 75us) server_negotiation.cc:299] Negotiation successful
0925 22:09:26.415721 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 22:09:26.425217 26853 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:09:26.425827 26853 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:09:26.434758 26853 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e62a2f376c0e46578af7df0333871c56: Generated new TSK 0
I20260925 22:09:26.434973 26853 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:09:26.437402 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 26773
W20260925 22:09:26.441929 26758 connection.cc:535] client connection to 127.5.0.126:37309 recv error: Network error: failed to read from TLS socket (remote: 127.5.0.126:37309): Connection reset by peer (error 104)
2026-09-25T22:09:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1314 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:09:26 dist-test-slave-7mqr krb5kdc[26872](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:09:26 dist-test-slave-7mqr krb5kdc[26872](info): set up 2 sockets
Sep 25 22:09:26 dist-test-slave-7mqr krb5kdc[26872](info): commencing operation
krb5kdc: starting...
W20260925 22:09:27.475870 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:09:27 dist-test-slave-7mqr krb5kdc[26872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:09:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:09:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.0.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
I20260925 22:09:27.594328 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:44447
--webserver_interface=127.5.0.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:40969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:44447
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:09:27.673875 26885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:27.674091 26885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:27.674115 26885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:27.675199 26885 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:09:27.675233 26885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:27.675247 26885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:27.675261 26885 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:09:27.675272 26885 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:09:27.676746 26885 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:40969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:44447
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:44447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:27.676940 26885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:27.677116 26885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:27.681957 26891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:27.681972 26890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:27.681981 26893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:27.682297 26885 server_base.cc:1034] running on GCE node
I20260925 22:09:27.682590 26885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:27.682888 26885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:27.684049 26885 hybrid_clock.cc:648] HybridClock initialized: now 1790374167684029 us; error 38 us; skew 500 ppm
Sep 25 22:09:27 dist-test-slave-7mqr krb5kdc[26872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374167, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:27.686812 26885 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:09:27.687290 26885 webserver.cc:460] Webserver started at http://127.5.0.126:41529/ using document root <none> and password file <none>
I20260925 22:09:27.687538 26885 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:27.687604 26885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:27.687747 26885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:27.688876 26885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "626118f10f4e4905b2b5e7404575e372"
format_stamp: "Formatted at 2026-09-25 22:09:27 on dist-test-slave-7mqr"
server_key: "e9ee4a49f27266f1cfe17a37b8d66149"
server_key_iv: "80d2305092cf44de266e71318f89109b"
server_key_version: "encryptionkey@0"
I20260925 22:09:27.689239 26885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "626118f10f4e4905b2b5e7404575e372"
format_stamp: "Formatted at 2026-09-25 22:09:27 on dist-test-slave-7mqr"
server_key: "e9ee4a49f27266f1cfe17a37b8d66149"
server_key_iv: "80d2305092cf44de266e71318f89109b"
server_key_version: "encryptionkey@0"
I20260925 22:09:27.690562 26885 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:09:27.691247 26900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:27.691424 26885 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:09:27.691536 26885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "626118f10f4e4905b2b5e7404575e372"
format_stamp: "Formatted at 2026-09-25 22:09:27 on dist-test-slave-7mqr"
server_key: "e9ee4a49f27266f1cfe17a37b8d66149"
server_key_iv: "80d2305092cf44de266e71318f89109b"
server_key_version: "encryptionkey@0"
I20260925 22:09:27.691610 26885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:27.695032 26885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:27.697479 26885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:27.697649 26885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:27.701820 26885 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:44447
I20260925 22:09:27.701855 26952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:44447 every 8 connection(s)
I20260925 22:09:27.702206 26885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:09:27.703260 26953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:27.705737 26953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 626118f10f4e4905b2b5e7404575e372: Bootstrap starting.
I20260925 22:09:27.706314 26953 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 626118f10f4e4905b2b5e7404575e372: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:27.706583 26953 log.cc:824] T 00000000000000000000000000000000 P 626118f10f4e4905b2b5e7404575e372: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:27.707269 26953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 626118f10f4e4905b2b5e7404575e372: No bootstrap required, opened a new log
I20260925 22:09:27.708679 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 26885
I20260925 22:09:27.708810 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:09:27.709009 5121 external_mini_cluster.cc:1428] Setting key c3c46063d8584cdbe5cb501d92fc4b63
I20260925 22:09:27.709193 26953 raft_consensus.cc:348] T 00000000000000000000000000000000 P 626118f10f4e4905b2b5e7404575e372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "626118f10f4e4905b2b5e7404575e372" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 44447 } }
I20260925 22:09:27.709343 26953 raft_consensus.cc:374] T 00000000000000000000000000000000 P 626118f10f4e4905b2b5e7404575e372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:27.709403 26953 raft_consensus.cc:729] T 00000000000000000000000000000000 P 626118f10f4e4905b2b5e7404575e372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 626118f10f4e4905b2b5e7404575e372, State: Initialized, Role: FOLLOWER
I20260925 22:09:27.709587 26953 consensus_queue.cc:260] T 00000000000000000000000000000000 P 626118f10f4e4905b2b5e7404575e372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "626118f10f4e4905b2b5e7404575e372" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 44447 } }
I20260925 22:09:27.709714 26953 raft_consensus.cc:388] T 00000000000000000000000000000000 P 626118f10f4e4905b2b5e7404575e372 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:09:27.709761 26953 raft_consensus.cc:482] T 00000000000000000000000000000000 P 626118f10f4e4905b2b5e7404575e372 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:09:27.709813 26953 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 626118f10f4e4905b2b5e7404575e372 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:27.710819 26953 raft_consensus.cc:504] T 00000000000000000000000000000000 P 626118f10f4e4905b2b5e7404575e372 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "626118f10f4e4905b2b5e7404575e372" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 44447 } }
I20260925 22:09:27.710937 26953 leader_election.cc:302] T 00000000000000000000000000000000 P 626118f10f4e4905b2b5e7404575e372 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 626118f10f4e4905b2b5e7404575e372; no voters:
I20260925 22:09:27.711125 26953 leader_election.cc:288] T 00000000000000000000000000000000 P 626118f10f4e4905b2b5e7404575e372 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:09:27.711259 26958 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 626118f10f4e4905b2b5e7404575e372 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:09:27.711344 26957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:27.709542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39111 (local address 127.5.0.126:44447)
0925 22:09:27.709743 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:27.709753 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:27.709787 (+ 34us) server_negotiation.cc:407] Connection header received
0925 22:09:27.709910 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:27.709917 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:27.709957 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:27.710089 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:27.710821 (+ 732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:27.710824 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:27.710825 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:27.710829 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:27.710877 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:27.710888 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:27.710963 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:27.711104 (+ 141us) server_negotiation.cc:299] Negotiation successful
0925 22:09:27.711147 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":92,"threads_started":1}
I20260925 22:09:27.711386 26953 sys_catalog.cc:564] T 00000000000000000000000000000000 P 626118f10f4e4905b2b5e7404575e372 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:09:27.712400 5121 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:09:27.711422 26958 raft_consensus.cc:686] T 00000000000000000000000000000000 P 626118f10f4e4905b2b5e7404575e372 [term 1 LEADER]: Becoming Leader. State: Replica: 626118f10f4e4905b2b5e7404575e372, State: Running, Role: LEADER
I20260925 22:09:27.712893 26958 consensus_queue.cc:237] T 00000000000000000000000000000000 P 626118f10f4e4905b2b5e7404575e372 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "626118f10f4e4905b2b5e7404575e372" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 44447 } }
I20260925 22:09:27.713482 26959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 626118f10f4e4905b2b5e7404575e372 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "626118f10f4e4905b2b5e7404575e372" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "626118f10f4e4905b2b5e7404575e372" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 44447 } } }
I20260925 22:09:27.713595 26959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 626118f10f4e4905b2b5e7404575e372 [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:27.713923 26976 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:09:27.713999 26960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 626118f10f4e4905b2b5e7404575e372 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 626118f10f4e4905b2b5e7404575e372. Latest consensus state: current_term: 1 leader_uuid: "626118f10f4e4905b2b5e7404575e372" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "626118f10f4e4905b2b5e7404575e372" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 44447 } } }
I20260925 22:09:27.714097 26960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 626118f10f4e4905b2b5e7404575e372 [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:27.714442 26976 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:09:27.715953 26976 catalog_manager.cc:1347] Generated new cluster ID: a756ad424cc54cc7b3f73f8a92f3025b
I20260925 22:09:27.716002 26976 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:09:27.720151 26957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:27.714305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52787 (local address 127.5.0.126:44447)
0925 22:09:27.714338 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:27.714339 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:27.715075 (+ 736us) server_negotiation.cc:407] Connection header received
0925 22:09:27.715143 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:27.715144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:27.715150 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:27.715208 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:27.715865 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:27.715867 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:27.715867 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:27.715871 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:27.715908 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:27.715913 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:27.715974 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:27.720065 (+ 4091us) server_negotiation.cc:299] Negotiation successful
0925 22:09:27.720080 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 22:09:27.731940 26976 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:09:27.732143 26976 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:09:27.736152 26976 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 626118f10f4e4905b2b5e7404575e372: Generated new TSK 0
I20260925 22:09:27.736334 26976 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:09:27.738610 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 26885
2026-09-25T22:09:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1300 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:09:27 dist-test-slave-7mqr krb5kdc[26984](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:09:27 dist-test-slave-7mqr krb5kdc[26984](info): set up 2 sockets
Sep 25 22:09:27 dist-test-slave-7mqr krb5kdc[26984](info): commencing operation
krb5kdc: starting...
W20260925 22:09:28.771986 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:09:28 dist-test-slave-7mqr krb5kdc[26984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:09:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:09:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.0.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
I20260925 22:09:28.890458 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:43933
--webserver_interface=127.5.0.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:44913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:43933
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:09:28.969909 26997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:28.970160 26997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:28.970197 26997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:28.971335 26997 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:09:28.971407 26997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:28.971431 26997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:28.971452 26997 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:09:28.971482 26997 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:09:28.972944 26997 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:44913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:43933
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:43933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:28.973220 26997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:28.973443 26997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:28.978276 27002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:28.978242 27005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:28.978259 26997 server_base.cc:1034] running on GCE node
W20260925 22:09:28.978223 27003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:28.978688 26997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:28.978987 26997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:28.980147 26997 hybrid_clock.cc:648] HybridClock initialized: now 1790374168980142 us; error 35 us; skew 500 ppm
Sep 25 22:09:28 dist-test-slave-7mqr krb5kdc[26984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374168, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:28.982527 26997 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:09:28.982990 26997 webserver.cc:460] Webserver started at http://127.5.0.126:38923/ using document root <none> and password file <none>
I20260925 22:09:28.983212 26997 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:28.983278 26997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:28.983426 26997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:28.984498 26997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "c34b6371a67f4543b5c9db1d34143f0f"
format_stamp: "Formatted at 2026-09-25 22:09:28 on dist-test-slave-7mqr"
server_key: "25e00528758f36682d483e754c723269"
server_key_iv: "b779906fa9d66defd7e70d4beb7c2138"
server_key_version: "encryptionkey@0"
I20260925 22:09:28.984845 26997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "c34b6371a67f4543b5c9db1d34143f0f"
format_stamp: "Formatted at 2026-09-25 22:09:28 on dist-test-slave-7mqr"
server_key: "25e00528758f36682d483e754c723269"
server_key_iv: "b779906fa9d66defd7e70d4beb7c2138"
server_key_version: "encryptionkey@0"
I20260925 22:09:28.986126 26997 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:09:28.986827 27012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:28.986994 26997 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:09:28.987110 26997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "c34b6371a67f4543b5c9db1d34143f0f"
format_stamp: "Formatted at 2026-09-25 22:09:28 on dist-test-slave-7mqr"
server_key: "25e00528758f36682d483e754c723269"
server_key_iv: "b779906fa9d66defd7e70d4beb7c2138"
server_key_version: "encryptionkey@0"
I20260925 22:09:28.987185 26997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:29.016511 26997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:29.018517 26997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:29.018688 26997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:29.022699 26997 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:43933
I20260925 22:09:29.022740 27064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:43933 every 8 connection(s)
I20260925 22:09:29.023118 26997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:09:29.024032 27065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:29.025426 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 26997
I20260925 22:09:29.025555 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:09:29.025748 5121 external_mini_cluster.cc:1428] Setting key 0fca2f025fa51c420762145f66581843
I20260925 22:09:29.026750 27065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c34b6371a67f4543b5c9db1d34143f0f: Bootstrap starting.
I20260925 22:09:29.027446 27065 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c34b6371a67f4543b5c9db1d34143f0f: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:29.027803 27065 log.cc:824] T 00000000000000000000000000000000 P c34b6371a67f4543b5c9db1d34143f0f: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:29.028550 27065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c34b6371a67f4543b5c9db1d34143f0f: No bootstrap required, opened a new log
I20260925 22:09:29.030805 27065 raft_consensus.cc:348] T 00000000000000000000000000000000 P c34b6371a67f4543b5c9db1d34143f0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34b6371a67f4543b5c9db1d34143f0f" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43933 } }
I20260925 22:09:29.030956 27065 raft_consensus.cc:374] T 00000000000000000000000000000000 P c34b6371a67f4543b5c9db1d34143f0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:29.031003 27065 raft_consensus.cc:729] T 00000000000000000000000000000000 P c34b6371a67f4543b5c9db1d34143f0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c34b6371a67f4543b5c9db1d34143f0f, State: Initialized, Role: FOLLOWER
I20260925 22:09:29.031141 27065 consensus_queue.cc:260] T 00000000000000000000000000000000 P c34b6371a67f4543b5c9db1d34143f0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34b6371a67f4543b5c9db1d34143f0f" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43933 } }
I20260925 22:09:29.031239 27065 raft_consensus.cc:388] T 00000000000000000000000000000000 P c34b6371a67f4543b5c9db1d34143f0f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:09:29.031281 27065 raft_consensus.cc:482] T 00000000000000000000000000000000 P c34b6371a67f4543b5c9db1d34143f0f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:09:29.031330 27065 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c34b6371a67f4543b5c9db1d34143f0f [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:29.031844 27065 raft_consensus.cc:504] T 00000000000000000000000000000000 P c34b6371a67f4543b5c9db1d34143f0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34b6371a67f4543b5c9db1d34143f0f" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43933 } }
I20260925 22:09:29.031994 27065 leader_election.cc:302] T 00000000000000000000000000000000 P c34b6371a67f4543b5c9db1d34143f0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c34b6371a67f4543b5c9db1d34143f0f; no voters:
I20260925 22:09:29.032186 27065 leader_election.cc:288] T 00000000000000000000000000000000 P c34b6371a67f4543b5c9db1d34143f0f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:09:29.032208 27070 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c34b6371a67f4543b5c9db1d34143f0f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:09:29.032327 27070 raft_consensus.cc:686] T 00000000000000000000000000000000 P c34b6371a67f4543b5c9db1d34143f0f [term 1 LEADER]: Becoming Leader. State: Replica: c34b6371a67f4543b5c9db1d34143f0f, State: Running, Role: LEADER
I20260925 22:09:29.032470 27065 sys_catalog.cc:564] T 00000000000000000000000000000000 P c34b6371a67f4543b5c9db1d34143f0f [sys.catalog]: configured and running, proceeding with master startup.
Sep 25 22:09:29 dist-test-slave-7mqr krb5kdc[26984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:09:29.032462 27070 consensus_queue.cc:237] T 00000000000000000000000000000000 P c34b6371a67f4543b5c9db1d34143f0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34b6371a67f4543b5c9db1d34143f0f" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43933 } }
I20260925 22:09:29.033089 27071 sys_catalog.cc:455] T 00000000000000000000000000000000 P c34b6371a67f4543b5c9db1d34143f0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c34b6371a67f4543b5c9db1d34143f0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34b6371a67f4543b5c9db1d34143f0f" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43933 } } }
I20260925 22:09:29.033316 27071 sys_catalog.cc:458] T 00000000000000000000000000000000 P c34b6371a67f4543b5c9db1d34143f0f [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:29.033236 27072 sys_catalog.cc:455] T 00000000000000000000000000000000 P c34b6371a67f4543b5c9db1d34143f0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c34b6371a67f4543b5c9db1d34143f0f. Latest consensus state: current_term: 1 leader_uuid: "c34b6371a67f4543b5c9db1d34143f0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34b6371a67f4543b5c9db1d34143f0f" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 43933 } } }
I20260925 22:09:29.033562 27072 sys_catalog.cc:458] T 00000000000000000000000000000000 P c34b6371a67f4543b5c9db1d34143f0f [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:29.033993 27077 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:09:29.034497 27077 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:09:29.035992 27077 catalog_manager.cc:1347] Generated new cluster ID: 29365ae0c29b495fa796d28dca5d2d89
I20260925 22:09:29.036043 27077 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:09:29.036899 27068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:29.026264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44571 (local address 127.5.0.126:43933)
0925 22:09:29.026443 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:29.026450 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:29.026475 (+ 25us) server_negotiation.cc:407] Connection header received
0925 22:09:29.026918 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:29.026925 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:29.026972 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:29.027092 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:29.027959 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:29.029005 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:29.030082 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:29.030458 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:29.030535 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:29.033329 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:29.033333 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:29.033333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:29.033345 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:29.035331 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:29.035715 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:29.035718 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:29.035718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:29.035819 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:29.036100 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:29.036101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:29.036101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:29.036339 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:29.036489 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:29.036609 (+ 120us) server_negotiation.cc:299] Negotiation successful
0925 22:09:29.036749 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":83,"threads_started":1}
I20260925 22:09:29.037773 5121 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:09:29.050058 27068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:29.040193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41845 (local address 127.5.0.126:43933)
0925 22:09:29.040230 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:29.040232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:29.044076 (+ 3844us) server_negotiation.cc:407] Connection header received
0925 22:09:29.044152 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:29.044153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:29.044163 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:29.044234 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:29.044745 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:29.045231 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:29.045947 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:29.046168 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:29.046247 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:29.047448 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:29.047449 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:29.047449 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:29.047456 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:29.048882 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:29.049283 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:29.049284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:29.049284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:29.049346 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:29.049550 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:29.049551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:29.049551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:29.049660 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:29.049775 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:29.049872 (+ 97us) server_negotiation.cc:299] Negotiation successful
0925 22:09:29.049950 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 22:09:29.051030 27077 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:09:29.051527 27077 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:09:29.055557 27077 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c34b6371a67f4543b5c9db1d34143f0f: Generated new TSK 0
I20260925 22:09:29.055745 27077 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:09:29.057979 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 26997
2026-09-25T22:09:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1319 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:09:29 dist-test-slave-7mqr krb5kdc[27096](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:09:29 dist-test-slave-7mqr krb5kdc[27096](info): set up 2 sockets
Sep 25 22:09:29 dist-test-slave-7mqr krb5kdc[27096](info): commencing operation
krb5kdc: starting...
W20260925 22:09:30.096082 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:09:30 dist-test-slave-7mqr krb5kdc[27096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:09:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:09:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.0.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
I20260925 22:09:30.213009 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:33953
--webserver_interface=127.5.0.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:43187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:33953
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:09:30.293331 27109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:30.293538 27109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:30.293560 27109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:30.294682 27109 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:09:30.294723 27109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:30.294736 27109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:30.294749 27109 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:09:30.294760 27109 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:09:30.296214 27109 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:43187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:33953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:33953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:30.296450 27109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:30.296703 27109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:30.301414 27114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:30.301415 27115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:30.301411 27117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:30.301702 27109 server_base.cc:1034] running on GCE node
I20260925 22:09:30.301934 27109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:30.302218 27109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:30.303381 27109 hybrid_clock.cc:648] HybridClock initialized: now 1790374170303369 us; error 30 us; skew 500 ppm
Sep 25 22:09:30 dist-test-slave-7mqr krb5kdc[27096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374170, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:30.305877 27109 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:09:30.306286 27109 webserver.cc:460] Webserver started at http://127.5.0.126:33695/ using document root <none> and password file <none>
I20260925 22:09:30.306521 27109 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:30.306591 27109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:30.306743 27109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:30.307761 27109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "8ae87fd9a62347168a5533bf8a36a000"
format_stamp: "Formatted at 2026-09-25 22:09:30 on dist-test-slave-7mqr"
server_key: "7afce59db45bb4ed14887c225dd0d799"
server_key_iv: "b26533da8c7bae052c051be0a05539ec"
server_key_version: "encryptionkey@0"
I20260925 22:09:30.308180 27109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "8ae87fd9a62347168a5533bf8a36a000"
format_stamp: "Formatted at 2026-09-25 22:09:30 on dist-test-slave-7mqr"
server_key: "7afce59db45bb4ed14887c225dd0d799"
server_key_iv: "b26533da8c7bae052c051be0a05539ec"
server_key_version: "encryptionkey@0"
I20260925 22:09:30.309686 27109 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:09:30.310395 27124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:30.310552 27109 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:09:30.310649 27109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "8ae87fd9a62347168a5533bf8a36a000"
format_stamp: "Formatted at 2026-09-25 22:09:30 on dist-test-slave-7mqr"
server_key: "7afce59db45bb4ed14887c225dd0d799"
server_key_iv: "b26533da8c7bae052c051be0a05539ec"
server_key_version: "encryptionkey@0"
I20260925 22:09:30.310736 27109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:30.323067 27109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:30.325485 27109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:30.325659 27109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:30.329648 27109 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:33953
I20260925 22:09:30.329687 27176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:33953 every 8 connection(s)
I20260925 22:09:30.330045 27109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:09:30.330874 27177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:30.333145 27177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ae87fd9a62347168a5533bf8a36a000: Bootstrap starting.
I20260925 22:09:30.333716 27177 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ae87fd9a62347168a5533bf8a36a000: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:30.334008 27177 log.cc:824] T 00000000000000000000000000000000 P 8ae87fd9a62347168a5533bf8a36a000: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:30.334703 27177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ae87fd9a62347168a5533bf8a36a000: No bootstrap required, opened a new log
I20260925 22:09:30.336326 27177 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ae87fd9a62347168a5533bf8a36a000 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ae87fd9a62347168a5533bf8a36a000" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 33953 } }
I20260925 22:09:30.336493 27177 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ae87fd9a62347168a5533bf8a36a000 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:30.336517 27177 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ae87fd9a62347168a5533bf8a36a000 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ae87fd9a62347168a5533bf8a36a000, State: Initialized, Role: FOLLOWER
I20260925 22:09:30.336686 27177 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ae87fd9a62347168a5533bf8a36a000 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ae87fd9a62347168a5533bf8a36a000" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 33953 } }
I20260925 22:09:30.336746 27177 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ae87fd9a62347168a5533bf8a36a000 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:09:30.336767 27177 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ae87fd9a62347168a5533bf8a36a000 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:09:30.336836 27177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ae87fd9a62347168a5533bf8a36a000 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:30.337548 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 27109
I20260925 22:09:30.337675 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:09:30.337692 27177 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ae87fd9a62347168a5533bf8a36a000 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ae87fd9a62347168a5533bf8a36a000" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 33953 } }
I20260925 22:09:30.337815 27177 leader_election.cc:302] T 00000000000000000000000000000000 P 8ae87fd9a62347168a5533bf8a36a000 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ae87fd9a62347168a5533bf8a36a000; no voters:
I20260925 22:09:30.337901 5121 external_mini_cluster.cc:1428] Setting key 50d6cfb79e719ec73ea2560877fafdb3
I20260925 22:09:30.337986 27177 leader_election.cc:288] T 00000000000000000000000000000000 P 8ae87fd9a62347168a5533bf8a36a000 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:09:30.338248 27177 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ae87fd9a62347168a5533bf8a36a000 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:09:30.338423 27180 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ae87fd9a62347168a5533bf8a36a000 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:09:30.338634 27180 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ae87fd9a62347168a5533bf8a36a000 [term 1 LEADER]: Becoming Leader. State: Replica: 8ae87fd9a62347168a5533bf8a36a000, State: Running, Role: LEADER
I20260925 22:09:30.338771 27180 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ae87fd9a62347168a5533bf8a36a000 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ae87fd9a62347168a5533bf8a36a000" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 33953 } }
I20260925 22:09:30.339236 27184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ae87fd9a62347168a5533bf8a36a000 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ae87fd9a62347168a5533bf8a36a000. Latest consensus state: current_term: 1 leader_uuid: "8ae87fd9a62347168a5533bf8a36a000" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ae87fd9a62347168a5533bf8a36a000" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 33953 } } }
I20260925 22:09:30.339323 27184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ae87fd9a62347168a5533bf8a36a000 [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:30.339530 27194 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:09:30.339776 27181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ae87fd9a62347168a5533bf8a36a000 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ae87fd9a62347168a5533bf8a36a000" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ae87fd9a62347168a5533bf8a36a000" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 33953 } } }
I20260925 22:09:30.339857 27181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ae87fd9a62347168a5533bf8a36a000 [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:30.340168 27194 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:09:30.341651 27194 catalog_manager.cc:1347] Generated new cluster ID: 255d25215a7e4196ae29a043f2907edb
I20260925 22:09:30.341702 27194 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:09:30.344403 27183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:30.338368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56565 (local address 127.5.0.126:33953)
0925 22:09:30.338719 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:30.338731 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:30.338775 (+ 44us) server_negotiation.cc:407] Connection header received
0925 22:09:30.338916 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:30.338923 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:30.339006 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:30.339169 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:30.339471 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:30.341038 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:30.342155 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:30.342783 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:30.342896 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:30.343591 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:30.343596 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:30.343596 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:30.343826 (+ 230us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:30.343869 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:30.343884 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:30.343976 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:30.344047 (+ 71us) server_negotiation.cc:299] Negotiation successful
0925 22:09:30.344110 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":66,"threads_started":1}
I20260925 22:09:30.348845 5121 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:09:30.353297 27194 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:09:30.353739 27183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:30.350314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41373 (local address 127.5.0.126:33953)
0925 22:09:30.350348 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:30.350350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:30.350514 (+ 164us) server_negotiation.cc:407] Connection header received
0925 22:09:30.350574 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:30.350574 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:30.350584 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:30.350635 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:30.350754 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:30.351465 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:30.352499 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:30.352781 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:30.352850 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:30.353496 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:30.353497 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:30.353497 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:30.353502 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:30.353515 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:30.353518 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:30.353567 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:30.353627 (+ 60us) server_negotiation.cc:299] Negotiation successful
0925 22:09:30.353648 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 22:09:30.354007 27194 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:09:30.361428 27194 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ae87fd9a62347168a5533bf8a36a000: Generated new TSK 0
I20260925 22:09:30.361604 27194 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:09:30.370231 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 27109
2026-09-25T22:09:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1311 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:09:30 dist-test-slave-7mqr krb5kdc[27208](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:09:30 dist-test-slave-7mqr krb5kdc[27208](info): set up 2 sockets
Sep 25 22:09:30 dist-test-slave-7mqr krb5kdc[27208](info): commencing operation
krb5kdc: starting...
W20260925 22:09:31.460079 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:09:31 dist-test-slave-7mqr krb5kdc[27208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:09:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:09:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.0.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
I20260925 22:09:31.579355 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:35553
--webserver_interface=127.5.0.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:38721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:35553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:09:31.662557 27221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:31.662752 27221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:31.662775 27221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:31.663825 27221 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:09:31.663858 27221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:31.663872 27221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:31.663900 27221 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:09:31.663914 27221 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:09:31.665336 27221 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:38721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:35553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:35553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:31.665511 27221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:31.665674 27221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:31.670377 27227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:31.670393 27229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:31.670382 27226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:31.670622 27221 server_base.cc:1034] running on GCE node
I20260925 22:09:31.670903 27221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:31.671218 27221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:31.672385 27221 hybrid_clock.cc:648] HybridClock initialized: now 1790374171672364 us; error 35 us; skew 500 ppm
Sep 25 22:09:31 dist-test-slave-7mqr krb5kdc[27208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374171, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:31.679172 27221 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:09:31.679584 27221 webserver.cc:460] Webserver started at http://127.5.0.126:41481/ using document root <none> and password file <none>
I20260925 22:09:31.679780 27221 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:31.679821 27221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:31.679983 27221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:31.681032 27221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "a14fdbc4cc964ee98f3cab52aa084100"
format_stamp: "Formatted at 2026-09-25 22:09:31 on dist-test-slave-7mqr"
server_key: "aedb88933758b22f55ba791a07858a75"
server_key_iv: "7c5f09bc113bc243ec13e591b4830dce"
server_key_version: "encryptionkey@0"
I20260925 22:09:31.681324 27221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "a14fdbc4cc964ee98f3cab52aa084100"
format_stamp: "Formatted at 2026-09-25 22:09:31 on dist-test-slave-7mqr"
server_key: "aedb88933758b22f55ba791a07858a75"
server_key_iv: "7c5f09bc113bc243ec13e591b4830dce"
server_key_version: "encryptionkey@0"
I20260925 22:09:31.682647 27221 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:09:31.683341 27236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:31.683581 27221 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:09:31.683669 27221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "a14fdbc4cc964ee98f3cab52aa084100"
format_stamp: "Formatted at 2026-09-25 22:09:31 on dist-test-slave-7mqr"
server_key: "aedb88933758b22f55ba791a07858a75"
server_key_iv: "7c5f09bc113bc243ec13e591b4830dce"
server_key_version: "encryptionkey@0"
I20260925 22:09:31.683724 27221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:31.686931 27221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:31.688933 27221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:31.689055 27221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:31.692742 27221 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:35553
I20260925 22:09:31.692798 27288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:35553 every 8 connection(s)
I20260925 22:09:31.693130 27221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:09:31.693719 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 27221
I20260925 22:09:31.693802 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:09:31.694036 5121 external_mini_cluster.cc:1428] Setting key 84f1a2b91d7298057f9053302dafa05f
I20260925 22:09:31.694271 27289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:31.696679 27292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:31.694573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47651 (local address 127.5.0.126:35553)
0925 22:09:31.694753 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:31.694761 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:31.694792 (+ 31us) server_negotiation.cc:407] Connection header received
0925 22:09:31.695396 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:31.695406 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:31.695440 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:31.695600 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:31.696257 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:31.696260 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:31.696261 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:31.696265 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:31.696312 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:31.696327 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:31.696407 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:31.696446 (+ 39us) server_negotiation.cc:299] Negotiation successful
0925 22:09:31.696483 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":78,"threads_started":1}
I20260925 22:09:31.697677 5121 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:09:31.697818 27289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a14fdbc4cc964ee98f3cab52aa084100: Bootstrap starting.
I20260925 22:09:31.698537 27289 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a14fdbc4cc964ee98f3cab52aa084100: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:31.698875 27289 log.cc:824] T 00000000000000000000000000000000 P a14fdbc4cc964ee98f3cab52aa084100: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:31.699674 27289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a14fdbc4cc964ee98f3cab52aa084100: No bootstrap required, opened a new log
I20260925 22:09:31.700438 27292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:31.699273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52853 (local address 127.5.0.126:35553)
0925 22:09:31.699328 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:31.699330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:31.699479 (+ 149us) server_negotiation.cc:407] Connection header received
0925 22:09:31.699534 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:31.699534 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:31.699538 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:31.699588 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:31.700213 (+ 625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:31.700214 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:31.700215 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:09:31.700218 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:31.700230 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:09:31.700234 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:31.700274 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:31.700337 (+ 63us) server_negotiation.cc:299] Negotiation successful
0925 22:09:31.700348 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 22:09:31.701874 27289 raft_consensus.cc:348] T 00000000000000000000000000000000 P a14fdbc4cc964ee98f3cab52aa084100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a14fdbc4cc964ee98f3cab52aa084100" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 35553 } }
I20260925 22:09:31.702014 27289 raft_consensus.cc:374] T 00000000000000000000000000000000 P a14fdbc4cc964ee98f3cab52aa084100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:09:31.702039 27289 raft_consensus.cc:729] T 00000000000000000000000000000000 P a14fdbc4cc964ee98f3cab52aa084100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a14fdbc4cc964ee98f3cab52aa084100, State: Initialized, Role: FOLLOWER
I20260925 22:09:31.702116 27289 consensus_queue.cc:260] T 00000000000000000000000000000000 P a14fdbc4cc964ee98f3cab52aa084100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a14fdbc4cc964ee98f3cab52aa084100" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 35553 } }
I20260925 22:09:31.702173 27289 raft_consensus.cc:388] T 00000000000000000000000000000000 P a14fdbc4cc964ee98f3cab52aa084100 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:09:31.702193 27289 raft_consensus.cc:482] T 00000000000000000000000000000000 P a14fdbc4cc964ee98f3cab52aa084100 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:09:31.702225 27289 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a14fdbc4cc964ee98f3cab52aa084100 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:31.702679 27289 raft_consensus.cc:504] T 00000000000000000000000000000000 P a14fdbc4cc964ee98f3cab52aa084100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a14fdbc4cc964ee98f3cab52aa084100" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 35553 } }
I20260925 22:09:31.702760 27289 leader_election.cc:302] T 00000000000000000000000000000000 P a14fdbc4cc964ee98f3cab52aa084100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a14fdbc4cc964ee98f3cab52aa084100; no voters:
I20260925 22:09:31.702876 27289 leader_election.cc:288] T 00000000000000000000000000000000 P a14fdbc4cc964ee98f3cab52aa084100 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:09:31.702952 27299 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a14fdbc4cc964ee98f3cab52aa084100 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:09:31.703099 27289 sys_catalog.cc:564] T 00000000000000000000000000000000 P a14fdbc4cc964ee98f3cab52aa084100 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:09:31.703119 27299 raft_consensus.cc:686] T 00000000000000000000000000000000 P a14fdbc4cc964ee98f3cab52aa084100 [term 1 LEADER]: Becoming Leader. State: Replica: a14fdbc4cc964ee98f3cab52aa084100, State: Running, Role: LEADER
I20260925 22:09:31.703204 27299 consensus_queue.cc:237] T 00000000000000000000000000000000 P a14fdbc4cc964ee98f3cab52aa084100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a14fdbc4cc964ee98f3cab52aa084100" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 35553 } }
I20260925 22:09:31.703608 27300 sys_catalog.cc:455] T 00000000000000000000000000000000 P a14fdbc4cc964ee98f3cab52aa084100 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a14fdbc4cc964ee98f3cab52aa084100" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a14fdbc4cc964ee98f3cab52aa084100" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 35553 } } }
I20260925 22:09:31.703606 27301 sys_catalog.cc:455] T 00000000000000000000000000000000 P a14fdbc4cc964ee98f3cab52aa084100 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a14fdbc4cc964ee98f3cab52aa084100. Latest consensus state: current_term: 1 leader_uuid: "a14fdbc4cc964ee98f3cab52aa084100" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a14fdbc4cc964ee98f3cab52aa084100" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 35553 } } }
I20260925 22:09:31.703722 27300 sys_catalog.cc:458] T 00000000000000000000000000000000 P a14fdbc4cc964ee98f3cab52aa084100 [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:31.703729 27301 sys_catalog.cc:458] T 00000000000000000000000000000000 P a14fdbc4cc964ee98f3cab52aa084100 [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:31.704293 27308 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:09:31.704784 27308 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:09:31.706229 27308 catalog_manager.cc:1347] Generated new cluster ID: 8dd6eeb2faa94092b33ade1b61315200
I20260925 22:09:31.706275 27308 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:09:31.716657 27308 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:09:31.716809 27308 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:09:31.723351 27308 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a14fdbc4cc964ee98f3cab52aa084100: Generated new TSK 0
I20260925 22:09:31.723515 27308 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:09:31.728189 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 27221
2026-09-25T22:09:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1358 ms)
[----------] 18 tests from AuthTokenIssuingTest (23763 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:09:31 dist-test-slave-7mqr krb5kdc[27320](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:09:31 dist-test-slave-7mqr krb5kdc[27320](info): set up 2 sockets
Sep 25 22:09:31 dist-test-slave-7mqr krb5kdc[27320](info): commencing operation
krb5kdc: starting...
W20260925 22:09:32.752229 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:09:32 dist-test-slave-7mqr krb5kdc[27320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:09:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:09:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.0.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
I20260925 22:09:32.870381 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:35921
--webserver_interface=127.5.0.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:33043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:35921,127.5.0.125:45031,127.5.0.124:44541
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:09:32.951493 27333 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 22:09:32.951737 27333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:32.951767 27333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:32.951810 27333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:32.952956 27333 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:09:32.953027 27333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:32.953051 27333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:32.953079 27333 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:09:32.953110 27333 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:09:32.954625 27333 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:33043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:35921,127.5.0.125:45031,127.5.0.124:44541
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:35921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:32.954912 27333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:32.955142 27333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:32.959836 27339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:32.959939 27338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:32.959978 27333 server_base.cc:1034] running on GCE node
W20260925 22:09:32.959836 27341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:32.960373 27333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:32.960676 27333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:32.961895 27333 hybrid_clock.cc:648] HybridClock initialized: now 1790374172961872 us; error 44 us; skew 500 ppm
Sep 25 22:09:32 dist-test-slave-7mqr krb5kdc[27320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374172, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:32.964470 27333 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:09:32.964958 27333 webserver.cc:460] Webserver started at http://127.5.0.126:40543/ using document root <none> and password file <none>
I20260925 22:09:32.965198 27333 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:32.965271 27333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:32.965423 27333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:32.966504 27333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "1a0f5f7b398f47f7a52bd0dc6b270082"
format_stamp: "Formatted at 2026-09-25 22:09:32 on dist-test-slave-7mqr"
server_key: "979ed82febf9030187bf58327161d865"
server_key_iv: "c0b8d76496bfd6900b60a00250922283"
server_key_version: "encryptionkey@0"
I20260925 22:09:32.966871 27333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "1a0f5f7b398f47f7a52bd0dc6b270082"
format_stamp: "Formatted at 2026-09-25 22:09:32 on dist-test-slave-7mqr"
server_key: "979ed82febf9030187bf58327161d865"
server_key_iv: "c0b8d76496bfd6900b60a00250922283"
server_key_version: "encryptionkey@0"
I20260925 22:09:32.968194 27333 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:09:32.968889 27348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:32.969054 27333 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:09:32.969161 27333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "1a0f5f7b398f47f7a52bd0dc6b270082"
format_stamp: "Formatted at 2026-09-25 22:09:32 on dist-test-slave-7mqr"
server_key: "979ed82febf9030187bf58327161d865"
server_key_iv: "c0b8d76496bfd6900b60a00250922283"
server_key_version: "encryptionkey@0"
I20260925 22:09:32.969233 27333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:32.977825 27333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:32.980108 27333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:32.980268 27333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:32.984233 27333 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:35921
I20260925 22:09:32.984263 27400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:35921 every 8 connection(s)
I20260925 22:09:32.984642 27333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:09:32.984927 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 27333
I20260925 22:09:32.985054 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:09:32.985266 5121 external_mini_cluster.cc:1428] Setting key bdb4f205c1d3292bad9572185b4bf24f
I20260925 22:09:32.985888 27401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:32.987772 27401 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.0.126" port: 35921 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.5.0.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.125@KRBTEST.COM" created.
Sep 25 22:09:32 dist-test-slave-7mqr krb5kdc[27320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374172, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:09:32.999785 27404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:32.988556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:35921 (local address 127.0.0.1:46950)
0925 22:09:32.988712 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:32.988760 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:32.989262 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:32.989564 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:32.989566 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:32.989844 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:32.989998 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:32.990002 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:32.991025 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:32.991026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:32.991702 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:32.991704 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:32.992317 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:32.992318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:32.992382 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:32.992976 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:09:32.992991 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:32.996031 (+ 3040us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:32.998264 (+ 2233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:32.998270 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:32.998272 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:32.998464 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:32.998730 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:32.998730 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:32.998730 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:32.998806 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:32.999300 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:32.999307 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:32.999372 (+ 65us) client_negotiation.cc:769] Sending connection context
0925 22:09:32.999472 (+ 100us) client_negotiation.cc:240] Negotiation successful
0925 22:09:32.999618 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":76,"threads_started":1}
I20260925 22:09:32.999882 27405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:32.988645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46950 (local address 127.5.0.126:35921)
0925 22:09:32.988790 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:32.988794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:32.988878 (+ 84us) server_negotiation.cc:407] Connection header received
0925 22:09:32.989423 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:32.989425 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:32.989460 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:32.989560 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:32.990099 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:32.990905 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:32.991806 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:32.992196 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:32.992328 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:32.996121 (+ 3793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:32.996128 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:32.996128 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:32.996135 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:32.998171 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:32.998548 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:32.998551 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:32.998551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:32.998659 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:32.998872 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:32.998873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:32.998874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:32.999206 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:32.999306 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:32.999528 (+ 222us) server_negotiation.cc:299] Negotiation successful
0925 22:09:32.999709 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":119,"thread_start_us":76,"threads_started":1}
I20260925 22:09:33.000828 27401 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.0.125" port: 45031 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:09:33.001199 27404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.001082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:45031 (local address 127.0.0.1:36174)
0925 22:09:33.001131 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:33.001179 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.125:45031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260925 22:09:33.001361 27349 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.0.125:45031: connect: Connection refused (error 111)
I20260925 22:09:33.001781 27404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.001699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:45031 (local address 127.0.0.1:36188)
0925 22:09:33.001741 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:33.001765 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.125:45031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260925 22:09:33.001912 27401 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.0.125:45031: Network error: Client connection negotiation failed: client connection to 127.5.0.125:45031: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.125.keytab.
Entry for principal kudu/127.5.0.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.125.keytab.
I20260925 22:09:33.011329 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.125:45031
--webserver_interface=127.5.0.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.125.keytab
--principal=kudu/127.5.0.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:33043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:35921,127.5.0.125:45031,127.5.0.124:44541
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:09:33.051169 27401 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.0.125" port: 45031 } attempt: 1
I20260925 22:09:33.051734 27404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.051579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:45031 (local address 127.0.0.1:36202)
0925 22:09:33.051645 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:33.051696 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.125:45031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260925 22:09:33.052198 27404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.052097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:45031 (local address 127.0.0.1:36214)
0925 22:09:33.052145 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:33.052173 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.125:45031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260925 22:09:33.052305 27401 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.0.125:45031: Network error: Client connection negotiation failed: client connection to 127.5.0.125:45031: connect: Connection refused (error 111)
W20260925 22:09:33.091526 27408 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 22:09:33.091728 27408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:33.091745 27408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:33.091765 27408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:33.092928 27408 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:09:33.092970 27408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:33.092988 27408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:33.093045 27408 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:09:33.093063 27408 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:09:33.094506 27408 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:33043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:35921,127.5.0.125:45031,127.5.0.124:44541
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.125.keytab
--principal=kudu/127.5.0.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.125:45031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.0.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:33.094758 27408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:33.094982 27408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:33.099717 27413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:33.099756 27408 server_base.cc:1034] running on GCE node
W20260925 22:09:33.099706 27416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:33.099706 27414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:33.100188 27408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:33.100454 27408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:33.101605 27408 hybrid_clock.cc:648] HybridClock initialized: now 1790374173101593 us; error 27 us; skew 500 ppm
Sep 25 22:09:33 dist-test-slave-7mqr krb5kdc[27320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374173, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:33.104100 27408 init.cc:377] Logged in from keytab as kudu/127.5.0.125@KRBTEST.COM (short username kudu)
I20260925 22:09:33.104508 27408 webserver.cc:460] Webserver started at http://127.5.0.125:40267/ using document root <none> and password file <none>
I20260925 22:09:33.104722 27408 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:33.104764 27408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:33.104902 27408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:33.105892 27408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-1/data/instance:
uuid: "98fd2a0c80e546dda0f72772edd2a4ba"
format_stamp: "Formatted at 2026-09-25 22:09:33 on dist-test-slave-7mqr"
server_key: "b6de48dea94634cd3837df87a38f388d"
server_key_iv: "0cb2d538f8b6d08fb3483d6468bb7e80"
server_key_version: "encryptionkey@0"
I20260925 22:09:33.106240 27408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-1/wal/instance:
uuid: "98fd2a0c80e546dda0f72772edd2a4ba"
format_stamp: "Formatted at 2026-09-25 22:09:33 on dist-test-slave-7mqr"
server_key: "b6de48dea94634cd3837df87a38f388d"
server_key_iv: "0cb2d538f8b6d08fb3483d6468bb7e80"
server_key_version: "encryptionkey@0"
I20260925 22:09:33.107537 27408 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 22:09:33.108289 27423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:33.108457 27408 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:33.108578 27408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-1/wal
uuid: "98fd2a0c80e546dda0f72772edd2a4ba"
format_stamp: "Formatted at 2026-09-25 22:09:33 on dist-test-slave-7mqr"
server_key: "b6de48dea94634cd3837df87a38f388d"
server_key_iv: "0cb2d538f8b6d08fb3483d6468bb7e80"
server_key_version: "encryptionkey@0"
I20260925 22:09:33.108659 27408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:33.120442 27401 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.0.125" port: 45031 } attempt: 2
I20260925 22:09:33.120913 27404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.120779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:45031 (local address 127.0.0.1:36228)
0925 22:09:33.120841 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:33.120880 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.125:45031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260925 22:09:33.121302 27404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.121219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:45031 (local address 127.0.0.1:36236)
0925 22:09:33.121261 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:33.121284 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.125:45031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260925 22:09:33.121390 27401 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.0.125:45031: Network error: Client connection negotiation failed: client connection to 127.5.0.125:45031: connect: Connection refused (error 111)
I20260925 22:09:33.125782 27408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:33.127861 27408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:33.128122 27408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:33.132321 27408 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.125:45031
I20260925 22:09:33.132404 27475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.125:45031 every 8 connection(s)
I20260925 22:09:33.132751 27408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-1/data/info.pb
I20260925 22:09:33.133790 27476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:33.135409 27476 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.0.126" port: 35921 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:09:33.136029 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 27408
I20260925 22:09:33.136143 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-1/wal/instance
I20260925 22:09:33.136339 5121 external_mini_cluster.cc:1428] Setting key 9cf462f4836c1ee7121df5ad89a512a7
Sep 25 22:09:33 dist-test-slave-7mqr krb5kdc[27320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374173, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.125@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.0.124@KRBTEST.COM; defaulting to no policy
I20260925 22:09:33.145452 27405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.136198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46962 (local address 127.5.0.126:35921)
0925 22:09:33.136260 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:33.136262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:33.136530 (+ 268us) server_negotiation.cc:407] Connection header received
0925 22:09:33.137028 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:33.137030 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:33.137043 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:33.137125 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:33.137771 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.138314 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.139423 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.139671 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.139771 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:33.142195 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:33.142196 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:33.142197 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:33.142206 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:33.143822 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:33.144331 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:33.144333 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:33.144333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:33.144400 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:33.144694 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:33.144695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:33.144696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:33.144851 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:33.144916 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:33.145257 (+ 341us) server_negotiation.cc:299] Negotiation successful
0925 22:09:33.145349 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.124@KRBTEST.COM" created.
I20260925 22:09:33.145725 27478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.136205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:35921 (local address 127.0.0.1:46962)
0925 22:09:33.136388 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:33.136433 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:33.136927 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:33.137180 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:33.137184 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:33.137449 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:33.137656 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:33.137661 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.138421 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.138423 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:33.139320 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:33.139322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.139795 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.139797 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:33.140010 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:33.140620 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:09:33.140629 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:33.142088 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:33.143949 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:33.143956 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:33.143958 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:33.144235 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:33.144474 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:33.144475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:33.144476 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:33.144605 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:33.144920 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:33.144927 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:33.145109 (+ 182us) client_negotiation.cc:769] Sending connection context
0925 22:09:33.145234 (+ 125us) client_negotiation.cc:240] Negotiation successful
0925 22:09:33.145453 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":103,"threads_started":1}
I20260925 22:09:33.146353 27476 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.0.125" port: 45031 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 22:09:33 dist-test-slave-7mqr krb5kdc[27320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374173, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.125@KRBTEST.COM for kudu/127.5.0.125@KRBTEST.COM
I20260925 22:09:33.154775 27478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.146695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:45031 (local address 127.0.0.1:36238)
0925 22:09:33.146774 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:33.146786 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:33.146840 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:33.147137 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:33.147139 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:33.147316 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:33.147365 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:33.147367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.148374 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.148375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:33.148921 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:33.148923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.149438 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.149439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:33.149511 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:33.149973 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:09:33.149979 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:33.151361 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:33.153725 (+ 2364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:33.153726 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:33.153726 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:33.153894 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:33.154170 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:33.154171 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:33.154171 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:33.154225 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:33.154525 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:33.154525 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:33.154565 (+ 40us) client_negotiation.cc:769] Sending connection context
0925 22:09:33.154622 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 22:09:33.154703 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260925 22:09:33.154893 27480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.146736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36238 (local address 127.5.0.125:45031)
0925 22:09:33.146957 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:33.146973 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:33.146998 (+ 25us) server_negotiation.cc:407] Connection header received
0925 22:09:33.147020 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:33.147022 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:33.147037 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:33.147112 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:33.147462 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.148280 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.149023 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.149356 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.149443 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:33.151451 (+ 2008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:33.151467 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:33.151468 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:33.151476 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:33.153620 (+ 2144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:33.154019 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:33.154024 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:33.154024 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:33.154090 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:33.154300 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:33.154300 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:33.154301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:33.154453 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:33.154530 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:33.154671 (+ 141us) server_negotiation.cc:299] Negotiation successful
0925 22:09:33.154769 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":194,"thread_start_us":91,"threads_started":1}
I20260925 22:09:33.155567 27476 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.0.124" port: 44541 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:09:33.156008 27478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.155860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:44541 (local address 127.0.0.1:38500)
0925 22:09:33.155940 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:33.155989 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.124:44541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260925 22:09:33.156194 27427 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.0.124:44541: connect: Connection refused (error 111)
I20260925 22:09:33.156685 27478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.156598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:44541 (local address 127.0.0.1:38502)
0925 22:09:33.156643 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:33.156669 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.124:44541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260925 22:09:33.156819 27476 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.0.124:44541: Network error: Client connection negotiation failed: client connection to 127.5.0.124:44541: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.124.keytab.
Entry for principal kudu/127.5.0.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.124.keytab.
I20260925 22:09:33.162683 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.124:44541
--webserver_interface=127.5.0.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.124.keytab
--principal=kudu/127.5.0.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:33043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:35921,127.5.0.125:45031,127.5.0.124:44541
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:09:33.206148 27476 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.0.124" port: 44541 } attempt: 1
I20260925 22:09:33.206732 27478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.206588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:44541 (local address 127.0.0.1:38506)
0925 22:09:33.206655 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:33.206701 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.124:44541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260925 22:09:33.207207 27478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.207081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:44541 (local address 127.0.0.1:38522)
0925 22:09:33.207151 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:33.207183 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.124:44541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260925 22:09:33.207346 27476 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.0.124:44541: Network error: Client connection negotiation failed: client connection to 127.5.0.124:44541: connect: Connection refused (error 111)
I20260925 22:09:33.212522 27401 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.0.125" port: 45031 } attempt: 3
Sep 25 22:09:33 dist-test-slave-7mqr krb5kdc[27320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374172, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for kudu/127.5.0.125@KRBTEST.COM
I20260925 22:09:33.219779 27480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.212928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36250 (local address 127.5.0.125:45031)
0925 22:09:33.212987 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:33.212995 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:33.213021 (+ 26us) server_negotiation.cc:407] Connection header received
0925 22:09:33.213081 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:33.213082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:33.213116 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:33.213181 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:33.213510 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.214015 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.214731 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.214985 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.215084 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:33.217238 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:33.217239 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:33.217239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:33.217245 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:33.218710 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:33.219100 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:33.219101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:33.219101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:33.219151 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:33.219340 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:33.219340 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:33.219340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:33.219438 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:33.219485 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:33.219652 (+ 167us) server_negotiation.cc:299] Negotiation successful
0925 22:09:33.219716 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 22:09:33.219830 27404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.212844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:45031 (local address 127.0.0.1:36250)
0925 22:09:33.212909 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:33.212920 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:33.213016 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:33.213188 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:33.213189 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:33.213358 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:33.213423 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:33.213425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.214107 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.214108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:33.214642 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:33.214643 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.215085 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.215086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:33.215172 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:33.215647 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:09:33.215657 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:33.217146 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:33.218805 (+ 1659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:33.218806 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:33.218807 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:33.219027 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:33.219215 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:33.219215 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:33.219216 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:33.219281 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:33.219496 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:33.219497 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:33.219551 (+ 54us) client_negotiation.cc:769] Sending connection context
0925 22:09:33.219628 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 22:09:33.219733 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 22:09:33.220237 27401 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.0.124" port: 44541 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:09:33.220607 27404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.220493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:44541 (local address 127.0.0.1:38536)
0925 22:09:33.220545 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:33.220580 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.124:44541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260925 22:09:33.220947 27404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.220857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:44541 (local address 127.0.0.1:38548)
0925 22:09:33.220903 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:33.220928 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.124:44541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260925 22:09:33.221087 27401 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.0.124:44541: Network error: Client connection negotiation failed: client connection to 127.5.0.124:44541: connect: Connection refused (error 111)
W20260925 22:09:33.247012 27483 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 22:09:33.247262 27483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:33.247293 27483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:33.247334 27483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:33.248486 27483 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:09:33.248558 27483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:33.248582 27483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:33.248610 27483 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:09:33.248633 27483 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:09:33.250145 27483 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:33043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:35921,127.5.0.125:45031,127.5.0.124:44541
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.124.keytab
--principal=kudu/127.5.0.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.124:44541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.0.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:33.250428 27483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:33.250707 27483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 22:09:33.252228 27401 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.0.124" port: 44541 } attempt: 1
I20260925 22:09:33.252655 27404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.252523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:44541 (local address 127.0.0.1:38562)
0925 22:09:33.252592 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:33.252626 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.124:44541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260925 22:09:33.253006 27404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.252912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:44541 (local address 127.0.0.1:38564)
0925 22:09:33.252957 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:33.252984 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.124:44541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260925 22:09:33.253091 27401 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.0.124:44541: Network error: Client connection negotiation failed: client connection to 127.5.0.124:44541: connect: Connection refused (error 111)
W20260925 22:09:33.255738 27491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:33.255685 27488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:33.255908 27483 server_base.cc:1034] running on GCE node
W20260925 22:09:33.255685 27489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:33.256227 27483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:33.256491 27483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:33.257642 27483 hybrid_clock.cc:648] HybridClock initialized: now 1790374173257621 us; error 37 us; skew 500 ppm
Sep 25 22:09:33 dist-test-slave-7mqr krb5kdc[27320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374173, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:33.260020 27483 init.cc:377] Logged in from keytab as kudu/127.5.0.124@KRBTEST.COM (short username kudu)
I20260925 22:09:33.260443 27483 webserver.cc:460] Webserver started at http://127.5.0.124:34127/ using document root <none> and password file <none>
I20260925 22:09:33.260679 27483 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:33.260722 27483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:33.260862 27483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:33.261938 27483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-2/data/instance:
uuid: "9c942fff072545a99aea4711922d5202"
format_stamp: "Formatted at 2026-09-25 22:09:33 on dist-test-slave-7mqr"
server_key: "29b083ae44b3e0b69c8d9a79638c5dfd"
server_key_iv: "7a4c1f20e4c0af0dbd6b798ab77bead4"
server_key_version: "encryptionkey@0"
I20260925 22:09:33.262324 27483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-2/wal/instance:
uuid: "9c942fff072545a99aea4711922d5202"
format_stamp: "Formatted at 2026-09-25 22:09:33 on dist-test-slave-7mqr"
server_key: "29b083ae44b3e0b69c8d9a79638c5dfd"
server_key_iv: "7a4c1f20e4c0af0dbd6b798ab77bead4"
server_key_version: "encryptionkey@0"
I20260925 22:09:33.263609 27483 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 22:09:33.264406 27498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:33.264601 27483 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:33.264686 27483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-2/wal
uuid: "9c942fff072545a99aea4711922d5202"
format_stamp: "Formatted at 2026-09-25 22:09:33 on dist-test-slave-7mqr"
server_key: "29b083ae44b3e0b69c8d9a79638c5dfd"
server_key_iv: "7a4c1f20e4c0af0dbd6b798ab77bead4"
server_key_version: "encryptionkey@0"
I20260925 22:09:33.264755 27483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:33.275571 27476 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.0.124" port: 44541 } attempt: 2
I20260925 22:09:33.276044 27478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.275863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:44541 (local address 127.0.0.1:38574)
0925 22:09:33.275974 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:33.276013 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.124:44541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260925 22:09:33.276401 27478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.276320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:44541 (local address 127.0.0.1:38582)
0925 22:09:33.276363 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:33.276386 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.124:44541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260925 22:09:33.276516 27476 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.0.124:44541: Network error: Client connection negotiation failed: client connection to 127.5.0.124:44541: connect: Connection refused (error 111)
I20260925 22:09:33.281240 27483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:33.283146 27483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:33.283301 27483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:33.287655 27483 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.124:44541
I20260925 22:09:33.287729 27550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.124:44541 every 8 connection(s)
I20260925 22:09:33.288136 27483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-2/data/info.pb
I20260925 22:09:33.289106 27551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:33.290654 27551 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.0.126" port: 35921 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 22:09:33 dist-test-slave-7mqr krb5kdc[27320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374173, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.124@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:09:33.297667 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 27483
I20260925 22:09:33.297811 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373908987421-5121-0/minicluster-data/master-2/wal/instance
I20260925 22:09:33.298004 5121 external_mini_cluster.cc:1428] Setting key 039aa9846e99ca9cb6a7b05349a677d7
I20260925 22:09:33.300232 27405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.291332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46978 (local address 127.5.0.126:35921)
0925 22:09:33.291399 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:33.291401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:33.291620 (+ 219us) server_negotiation.cc:407] Connection header received
0925 22:09:33.292165 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:33.292166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:33.292176 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:33.292237 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:33.292861 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.293327 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.294308 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.294511 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.294580 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:33.296946 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:33.296947 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:33.296947 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:33.296954 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:33.298693 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:33.299199 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:33.299200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:33.299201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:33.299252 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:33.299565 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:33.299566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:33.299566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:33.299699 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:33.299767 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:33.300068 (+ 301us) server_negotiation.cc:299] Negotiation successful
0925 22:09:33.300146 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 22:09:33.300326 27553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.291297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:35921 (local address 127.0.0.1:46978)
0925 22:09:33.291481 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:33.291524 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:33.292076 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:33.292311 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:33.292314 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:33.292589 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:33.292789 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:33.292793 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.293402 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.293402 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:33.294203 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:33.294205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.294607 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.294608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:33.294713 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:33.295233 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:09:33.295241 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:33.296863 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:33.298785 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:33.298791 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:33.298793 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:33.299124 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:33.299319 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:33.299320 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:33.299320 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:33.299488 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:33.299765 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:33.299770 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:33.299885 (+ 115us) client_negotiation.cc:769] Sending connection context
0925 22:09:33.300020 (+ 135us) client_negotiation.cc:240] Negotiation successful
0925 22:09:33.300139 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":103,"threads_started":1}
I20260925 22:09:33.301136 27551 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.0.125" port: 45031 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 22:09:33 dist-test-slave-7mqr krb5kdc[27320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.124@KRBTEST.COM
Sep 25 22:09:33 dist-test-slave-7mqr krb5kdc[27320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374173, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.124@KRBTEST.COM for kudu/127.5.0.125@KRBTEST.COM
I20260925 22:09:33.307463 27555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.298408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38584 (local address 127.5.0.124:44541)
0925 22:09:33.298626 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:33.298632 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:33.298659 (+ 27us) server_negotiation.cc:407] Connection header received
0925 22:09:33.298689 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:33.298691 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:33.298706 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:33.298781 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:33.299586 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.300666 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.301608 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.301915 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.301995 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:33.304503 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:33.304510 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:33.304511 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:33.304518 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:33.306127 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:33.306526 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:33.306529 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:33.306529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:33.306596 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:33.306814 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:33.306814 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:33.306815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:33.306970 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:33.307136 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:33.307268 (+ 132us) server_negotiation.cc:299] Negotiation successful
0925 22:09:33.307348 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":107,"threads_started":1}
I20260925 22:09:33.307818 27553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.301386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:45031 (local address 127.0.0.1:36256)
0925 22:09:33.301426 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:33.301435 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:33.301466 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:33.301612 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:33.301613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:33.301750 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:33.301792 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:33.301793 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.302414 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.302415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:33.302906 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:33.302908 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.303288 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.303289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:33.303355 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:33.303742 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:09:33.303746 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:33.305150 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:33.306755 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:33.306757 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:33.306757 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:33.306923 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:33.307154 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:33.307154 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:33.307155 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:33.307217 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:33.307515 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:33.307516 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:33.307558 (+ 42us) client_negotiation.cc:769] Sending connection context
0925 22:09:33.307639 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 22:09:33.307731 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260925 22:09:33.307837 27480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.301438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36256 (local address 127.5.0.125:45031)
0925 22:09:33.301486 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:33.301487 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:33.301510 (+ 23us) server_negotiation.cc:407] Connection header received
0925 22:09:33.301539 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:33.301539 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:33.301547 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:33.301607 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:33.301871 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.302319 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.302996 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.303218 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.303292 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:33.305244 (+ 1952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:33.305246 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:33.305246 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:33.305253 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:33.306669 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:33.307001 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:33.307003 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:33.307003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:33.307078 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:33.307298 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:33.307301 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:33.307301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:33.307406 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:33.307483 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:33.307677 (+ 194us) server_negotiation.cc:299] Negotiation successful
0925 22:09:33.307750 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 22:09:33.308559 27551 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.0.124" port: 44541 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 22:09:33 dist-test-slave-7mqr krb5kdc[27320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374173, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.124@KRBTEST.COM for kudu/127.5.0.124@KRBTEST.COM
Sep 25 22:09:33 dist-test-slave-7mqr krb5kdc[27320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:09:33.315554 27553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.308831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:44541 (local address 127.0.0.1:38600)
0925 22:09:33.308866 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:33.308878 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:33.308948 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:33.309113 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:33.309114 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:33.309240 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:33.309264 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:33.309265 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.309881 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.309882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:33.310360 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:33.310361 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.310804 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.310805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:33.310854 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:33.311269 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:09:33.311275 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:33.312798 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:33.314531 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:33.314532 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:33.314532 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:33.314680 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:33.314893 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:33.314894 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:33.314894 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:33.314951 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:33.315194 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:33.315195 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:33.315233 (+ 38us) client_negotiation.cc:769] Sending connection context
0925 22:09:33.315299 (+ 66us) client_negotiation.cc:240] Negotiation successful
0925 22:09:33.315397 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260925 22:09:33.315557 27555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.308899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38600 (local address 127.5.0.124:44541)
0925 22:09:33.309015 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:33.309017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:33.309022 (+ 5us) server_negotiation.cc:407] Connection header received
0925 22:09:33.309037 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:33.309037 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:33.309045 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:33.309110 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:33.309333 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.309783 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.310447 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.310722 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.310805 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:33.312899 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:33.312900 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:33.312900 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:33.312907 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:33.314450 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:33.314771 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:33.314772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:33.314773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:33.314822 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:33.315019 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:33.315020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:33.315021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:33.315131 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:33.315204 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:33.315327 (+ 123us) server_negotiation.cc:299] Negotiation successful
0925 22:09:33.315441 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":101}
I20260925 22:09:33.316133 27405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.308278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46982 (local address 127.5.0.126:35921)
0925 22:09:33.308315 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:33.308317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:33.308327 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:09:33.308348 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:33.308349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:33.308358 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:33.308417 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:33.309086 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.309851 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.310511 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.310749 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.310824 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:33.313509 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:33.313511 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:33.313511 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:33.313519 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:33.314947 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:33.315287 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:33.315288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:33.315288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:33.315348 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:33.315564 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:33.315564 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:33.315565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:33.315676 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:33.315871 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:33.315926 (+ 55us) server_negotiation.cc:299] Negotiation successful
0925 22:09:33.316007 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 22:09:33.317703 27551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c942fff072545a99aea4711922d5202: Bootstrap starting.
I20260925 22:09:33.318691 27551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c942fff072545a99aea4711922d5202: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:33.319006 27551 log.cc:824] T 00000000000000000000000000000000 P 9c942fff072545a99aea4711922d5202: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:33.319994 27551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c942fff072545a99aea4711922d5202: No bootstrap required, opened a new log
Sep 25 22:09:33 dist-test-slave-7mqr krb5kdc[27320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.125@KRBTEST.COM
I20260925 22:09:33.321841 27551 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c942fff072545a99aea4711922d5202 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f5f7b398f47f7a52bd0dc6b270082" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 35921 } } peers { permanent_uuid: "98fd2a0c80e546dda0f72772edd2a4ba" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 45031 } } peers { permanent_uuid: "9c942fff072545a99aea4711922d5202" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 44541 } }
I20260925 22:09:33.321987 27551 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c942fff072545a99aea4711922d5202 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c942fff072545a99aea4711922d5202, State: Initialized, Role: FOLLOWER
I20260925 22:09:33.322080 27551 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c942fff072545a99aea4711922d5202 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f5f7b398f47f7a52bd0dc6b270082" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 35921 } } peers { permanent_uuid: "98fd2a0c80e546dda0f72772edd2a4ba" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 45031 } } peers { permanent_uuid: "9c942fff072545a99aea4711922d5202" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 44541 } }
I20260925 22:09:33.322419 27551 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c942fff072545a99aea4711922d5202 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:09:33.322386 27557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c942fff072545a99aea4711922d5202 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f5f7b398f47f7a52bd0dc6b270082" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 35921 } } peers { permanent_uuid: "98fd2a0c80e546dda0f72772edd2a4ba" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 45031 } } peers { permanent_uuid: "9c942fff072545a99aea4711922d5202" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 44541 } } }
I20260925 22:09:33.322511 27557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c942fff072545a99aea4711922d5202 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 22:09:33.324421 27568 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9c942fff072545a99aea4711922d5202: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:09:33.324723 27568 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:09:33.324957 27480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.316781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36262 (local address 127.5.0.125:45031)
0925 22:09:33.316827 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:33.316828 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:33.316878 (+ 50us) server_negotiation.cc:407] Connection header received
0925 22:09:33.316949 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:33.316950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:33.316958 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:33.317002 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:33.317565 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.318074 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.318996 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.319220 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.319271 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:33.321747 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:33.321749 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:33.321749 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:33.321756 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:33.323497 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:33.323822 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:33.323823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:33.323824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:33.323905 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:33.324101 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:33.324102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:33.324102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:33.324236 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:33.324381 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:33.324791 (+ 410us) server_negotiation.cc:299] Negotiation successful
0925 22:09:33.324867 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 22:09:33.326416 5121 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:09:33.324770 27483 master_runner.cc:186] Error getting master registration for 127.5.0.126:35921: OK, instance_id { permanent_uuid: "1a0f5f7b398f47f7a52bd0dc6b270082" instance_seqno: 1790374172983089 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 22:09:33.327152 27483 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 22:09:33.327373 27375 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "1a0f5f7b398f47f7a52bd0dc6b270082"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46982
I20260925 22:09:33.328225 27401 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.0.124" port: 44541 } attempt: 2
Sep 25 22:09:33 dist-test-slave-7mqr krb5kdc[27320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374172, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for kudu/127.5.0.124@KRBTEST.COM
I20260925 22:09:33.335829 27404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.328456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:44541 (local address 127.0.0.1:38616)
0925 22:09:33.328516 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:33.328530 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:33.328576 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:33.328704 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:33.328705 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:33.328868 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:33.328911 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:33.328914 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.329768 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.329769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:33.330583 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:33.330586 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.331041 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.331042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:33.331111 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:33.331547 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:09:33.331552 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:33.333009 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:33.334723 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:33.334724 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:33.334725 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:33.334939 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:33.335158 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:33.335159 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:33.335159 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:33.335224 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:33.335513 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:33.335513 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:33.335560 (+ 47us) client_negotiation.cc:769] Sending connection context
0925 22:09:33.335637 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 22:09:33.335729 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260925 22:09:33.335834 27555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.328521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38616 (local address 127.5.0.124:44541)
0925 22:09:33.328572 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:33.328574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:33.328582 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:09:33.328639 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:33.328639 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:33.328648 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:33.328703 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:33.328988 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.329686 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.330668 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.330955 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.331023 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:33.333117 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:33.333118 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:33.333119 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:33.333126 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:33.334639 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:33.335016 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:33.335017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:33.335017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:33.335079 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:33.335290 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:33.335290 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:33.335291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:33.335443 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:33.335506 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:33.335667 (+ 161us) server_negotiation.cc:299] Negotiation successful
0925 22:09:33.335743 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 22:09:33.337595 27401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a0f5f7b398f47f7a52bd0dc6b270082: Bootstrap starting.
I20260925 22:09:33.338313 27401 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a0f5f7b398f47f7a52bd0dc6b270082: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:33.338604 27401 log.cc:824] T 00000000000000000000000000000000 P 1a0f5f7b398f47f7a52bd0dc6b270082: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:33.339447 27401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a0f5f7b398f47f7a52bd0dc6b270082: No bootstrap required, opened a new log
I20260925 22:09:33.340816 27401 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a0f5f7b398f47f7a52bd0dc6b270082 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f5f7b398f47f7a52bd0dc6b270082" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 35921 } } peers { permanent_uuid: "98fd2a0c80e546dda0f72772edd2a4ba" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 45031 } } peers { permanent_uuid: "9c942fff072545a99aea4711922d5202" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 44541 } }
I20260925 22:09:33.340955 27401 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a0f5f7b398f47f7a52bd0dc6b270082 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a0f5f7b398f47f7a52bd0dc6b270082, State: Initialized, Role: FOLLOWER
I20260925 22:09:33.341035 27401 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a0f5f7b398f47f7a52bd0dc6b270082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f5f7b398f47f7a52bd0dc6b270082" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 35921 } } peers { permanent_uuid: "98fd2a0c80e546dda0f72772edd2a4ba" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 45031 } } peers { permanent_uuid: "9c942fff072545a99aea4711922d5202" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 44541 } }
I20260925 22:09:33.341312 27571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a0f5f7b398f47f7a52bd0dc6b270082 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f5f7b398f47f7a52bd0dc6b270082" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 35921 } } peers { permanent_uuid: "98fd2a0c80e546dda0f72772edd2a4ba" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 45031 } } peers { permanent_uuid: "9c942fff072545a99aea4711922d5202" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 44541 } } }
I20260925 22:09:33.341404 27375 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a0f5f7b398f47f7a52bd0dc6b270082 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260925 22:09:33.341450 27571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a0f5f7b398f47f7a52bd0dc6b270082 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:09:33.341360 27401 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a0f5f7b398f47f7a52bd0dc6b270082 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:09:33.341511 27375 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a0f5f7b398f47f7a52bd0dc6b270082 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:33.342440 27375 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a0f5f7b398f47f7a52bd0dc6b270082 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f5f7b398f47f7a52bd0dc6b270082" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 35921 } } peers { permanent_uuid: "98fd2a0c80e546dda0f72772edd2a4ba" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 45031 } } peers { permanent_uuid: "9c942fff072545a99aea4711922d5202" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 44541 } }
I20260925 22:09:33.343010 27525 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a0f5f7b398f47f7a52bd0dc6b270082" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9c942fff072545a99aea4711922d5202"
I20260925 22:09:33.343130 27525 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c942fff072545a99aea4711922d5202 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:33.344138 27525 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9c942fff072545a99aea4711922d5202 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a0f5f7b398f47f7a52bd0dc6b270082 in term 1.
I20260925 22:09:33.344404 27352 leader_election.cc:302] T 00000000000000000000000000000000 P 1a0f5f7b398f47f7a52bd0dc6b270082 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a0f5f7b398f47f7a52bd0dc6b270082, 9c942fff072545a99aea4711922d5202; no voters:
I20260925 22:09:33.344573 27571 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a0f5f7b398f47f7a52bd0dc6b270082 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:09:33.344712 27571 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a0f5f7b398f47f7a52bd0dc6b270082 [term 1 LEADER]: Becoming Leader. State: Replica: 1a0f5f7b398f47f7a52bd0dc6b270082, State: Running, Role: LEADER
I20260925 22:09:33.344798 27571 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a0f5f7b398f47f7a52bd0dc6b270082 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f5f7b398f47f7a52bd0dc6b270082" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 35921 } } peers { permanent_uuid: "98fd2a0c80e546dda0f72772edd2a4ba" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 45031 } } peers { permanent_uuid: "9c942fff072545a99aea4711922d5202" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 44541 } }
I20260925 22:09:33.345221 27571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a0f5f7b398f47f7a52bd0dc6b270082 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a0f5f7b398f47f7a52bd0dc6b270082. Latest consensus state: current_term: 1 leader_uuid: "1a0f5f7b398f47f7a52bd0dc6b270082" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f5f7b398f47f7a52bd0dc6b270082" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 35921 } } peers { permanent_uuid: "98fd2a0c80e546dda0f72772edd2a4ba" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 45031 } } peers { permanent_uuid: "9c942fff072545a99aea4711922d5202" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 44541 } } }
I20260925 22:09:33.345367 27571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a0f5f7b398f47f7a52bd0dc6b270082 [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:33.344046 27375 leader_election.cc:288] T 00000000000000000000000000000000 P 1a0f5f7b398f47f7a52bd0dc6b270082 [CANDIDATE]: Term 1 election: Requested vote from peers 98fd2a0c80e546dda0f72772edd2a4ba (127.5.0.125:45031), 9c942fff072545a99aea4711922d5202 (127.5.0.124:44541)
W20260925 22:09:33.343739 27582 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a0f5f7b398f47f7a52bd0dc6b270082: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:09:33.345506 27582 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:09:33.345218 27333 master_runner.cc:186] Error getting master registration for 127.5.0.125:45031: OK, instance_id { permanent_uuid: "98fd2a0c80e546dda0f72772edd2a4ba" instance_seqno: 1790374173131077 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 22:09:33.347369 27333 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 22:09:33.347608 27586 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:09:33.347841 27586 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:09:33.349522 27525 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9c942fff072545a99aea4711922d5202 [term 1 FOLLOWER]: Refusing update from remote peer 1a0f5f7b398f47f7a52bd0dc6b270082: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:09:33.349864 27571 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1a0f5f7b398f47f7a52bd0dc6b270082 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c942fff072545a99aea4711922d5202" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:09:33.351081 27557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c942fff072545a99aea4711922d5202 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a0f5f7b398f47f7a52bd0dc6b270082. Latest consensus state: current_term: 1 leader_uuid: "1a0f5f7b398f47f7a52bd0dc6b270082" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f5f7b398f47f7a52bd0dc6b270082" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 35921 } } peers { permanent_uuid: "98fd2a0c80e546dda0f72772edd2a4ba" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 45031 } } peers { permanent_uuid: "9c942fff072545a99aea4711922d5202" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 44541 } } }
I20260925 22:09:33.351161 27557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c942fff072545a99aea4711922d5202 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:09:33.352489 27571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a0f5f7b398f47f7a52bd0dc6b270082 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1a0f5f7b398f47f7a52bd0dc6b270082" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f5f7b398f47f7a52bd0dc6b270082" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 35921 } } peers { permanent_uuid: "98fd2a0c80e546dda0f72772edd2a4ba" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 45031 } } peers { permanent_uuid: "9c942fff072545a99aea4711922d5202" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 44541 } } }
I20260925 22:09:33.352583 27571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a0f5f7b398f47f7a52bd0dc6b270082 [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:33.352588 27557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c942fff072545a99aea4711922d5202 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1a0f5f7b398f47f7a52bd0dc6b270082" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f5f7b398f47f7a52bd0dc6b270082" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 35921 } } peers { permanent_uuid: "98fd2a0c80e546dda0f72772edd2a4ba" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 45031 } } peers { permanent_uuid: "9c942fff072545a99aea4711922d5202" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 44541 } } }
I20260925 22:09:33.352671 27557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c942fff072545a99aea4711922d5202 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:09:33.353272 27593 mvcc.cc:205] Tried to move back new op lower bound from 7333372614036987904 to 7333372614020993024. Current Snapshot: MvccSnapshot[applied={T|T < 7333372614036987904}]
I20260925 22:09:33.360354 27586 catalog_manager.cc:1347] Generated new cluster ID: 356ab715ca4d4fafa5fc2c255c90e3b5
I20260925 22:09:33.360404 27586 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:09:33.361016 27480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.348717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36276 (local address 127.5.0.125:45031)
0925 22:09:33.348759 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:33.348761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:33.349009 (+ 248us) server_negotiation.cc:407] Connection header received
0925 22:09:33.349113 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:33.349114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:33.349125 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:33.349157 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:33.350683 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.351378 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.352867 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.353166 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.353217 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:33.354839 (+ 1622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:33.354840 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:33.354841 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:33.354848 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:33.356631 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:33.357050 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:33.357051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:33.357051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:33.357109 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:33.357306 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:33.357307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:33.357308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:33.357452 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:33.357549 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:33.360815 (+ 3266us) server_negotiation.cc:299] Negotiation successful
0925 22:09:33.360893 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:09:33.366940 27555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.356194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38620 (local address 127.5.0.124:44541)
0925 22:09:33.356229 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:33.356231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:33.358017 (+ 1786us) server_negotiation.cc:407] Connection header received
0925 22:09:33.358039 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:33.358039 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:33.358049 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:33.358097 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:33.358708 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.359375 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.360327 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.360623 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.360678 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:33.363130 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:33.363132 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:33.363132 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:33.363139 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:33.365381 (+ 2242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:33.365905 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:33.365906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:33.365907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:33.365971 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:33.366169 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:33.366170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:33.366170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:33.366300 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:33.366419 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:33.366753 (+ 334us) server_negotiation.cc:299] Negotiation successful
0925 22:09:33.366833 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 22:09:33.367230 27405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.355031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46996 (local address 127.5.0.126:35921)
0925 22:09:33.355321 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:33.355323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:33.356798 (+ 1475us) server_negotiation.cc:407] Connection header received
0925 22:09:33.356965 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:33.356965 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:33.356992 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:33.357024 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:33.358259 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.359016 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.360647 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.360924 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.360965 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:33.363451 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:33.363452 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:33.363452 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:33.363480 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:33.365224 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:33.365946 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:33.365947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:33.365947 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:33.365998 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:33.366319 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:33.366320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:33.366320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:33.366492 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:33.366576 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:33.367072 (+ 496us) server_negotiation.cc:299] Negotiation successful
0925 22:09:33.367144 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 22:09:33.367676 27476 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.0.124" port: 44541 } attempt: 3
Sep 25 22:09:33 dist-test-slave-7mqr krb5kdc[27320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374173, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.125@KRBTEST.COM for kudu/127.5.0.124@KRBTEST.COM
I20260925 22:09:33.375674 27555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.368083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38622 (local address 127.5.0.124:44541)
0925 22:09:33.368139 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:33.368140 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:33.368147 (+ 7us) server_negotiation.cc:407] Connection header received
0925 22:09:33.368166 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:33.368167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:33.368200 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:33.368594 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:33.368651 (+ 57us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.369376 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.370036 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.370327 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.370385 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:33.372442 (+ 2057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:33.372444 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:33.372444 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:33.372450 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:33.374196 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:33.374843 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:33.374844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:33.374844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:33.374898 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:33.375155 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:33.375155 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:33.375156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:33.375299 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:33.375352 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:33.375519 (+ 167us) server_negotiation.cc:299] Negotiation successful
0925 22:09:33.375587 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 22:09:33.375677 27478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:33.367908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:44541 (local address 127.0.0.1:38622)
0925 22:09:33.367947 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:33.367957 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:33.368017 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:33.368305 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:33.368307 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:33.368469 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:33.368510 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:33.368512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.369445 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.369446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:33.369938 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:33.369939 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:33.370389 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:33.370390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:33.370443 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:33.370846 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:09:33.370850 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:33.372345 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:33.374282 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:33.374283 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:33.374283 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:33.374511 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:33.374999 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:33.375000 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:33.375001 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:33.375082 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:33.375364 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:33.375365 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:33.375404 (+ 39us) client_negotiation.cc:769] Sending connection context
0925 22:09:33.375495 (+ 91us) client_negotiation.cc:240] Negotiation successful
0925 22:09:33.375587 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260925 22:09:33.377593 27476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98fd2a0c80e546dda0f72772edd2a4ba: Bootstrap starting.
I20260925 22:09:33.378350 27476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98fd2a0c80e546dda0f72772edd2a4ba: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:33.378651 27476 log.cc:824] T 00000000000000000000000000000000 P 98fd2a0c80e546dda0f72772edd2a4ba: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:33.380090 27476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98fd2a0c80e546dda0f72772edd2a4ba: No bootstrap required, opened a new log
I20260925 22:09:33.382068 27476 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98fd2a0c80e546dda0f72772edd2a4ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f5f7b398f47f7a52bd0dc6b270082" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 35921 } } peers { permanent_uuid: "98fd2a0c80e546dda0f72772edd2a4ba" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 45031 } } peers { permanent_uuid: "9c942fff072545a99aea4711922d5202" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 44541 } }
I20260925 22:09:33.382202 27476 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98fd2a0c80e546dda0f72772edd2a4ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98fd2a0c80e546dda0f72772edd2a4ba, State: Initialized, Role: FOLLOWER
I20260925 22:09:33.382289 27476 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98fd2a0c80e546dda0f72772edd2a4ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f5f7b398f47f7a52bd0dc6b270082" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 35921 } } peers { permanent_uuid: "98fd2a0c80e546dda0f72772edd2a4ba" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 45031 } } peers { permanent_uuid: "9c942fff072545a99aea4711922d5202" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 44541 } }
I20260925 22:09:33.382642 27476 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98fd2a0c80e546dda0f72772edd2a4ba [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:09:33.382771 27449 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98fd2a0c80e546dda0f72772edd2a4ba [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:33.383602 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a0f5f7b398f47f7a52bd0dc6b270082" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "98fd2a0c80e546dda0f72772edd2a4ba"
I20260925 22:09:33.383720 27449 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 98fd2a0c80e546dda0f72772edd2a4ba [term 1 FOLLOWER]: Refusing update from remote peer 1a0f5f7b398f47f7a52bd0dc6b270082: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:09:33.384140 27584 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1a0f5f7b398f47f7a52bd0dc6b270082 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98fd2a0c80e546dda0f72772edd2a4ba" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 45031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:09:33.382596 27600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98fd2a0c80e546dda0f72772edd2a4ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f5f7b398f47f7a52bd0dc6b270082" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 35921 } } peers { permanent_uuid: "98fd2a0c80e546dda0f72772edd2a4ba" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 45031 } } peers { permanent_uuid: "9c942fff072545a99aea4711922d5202" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 44541 } } }
I20260925 22:09:33.384842 27600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98fd2a0c80e546dda0f72772edd2a4ba [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:09:33.384969 27450 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 98fd2a0c80e546dda0f72772edd2a4ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a0f5f7b398f47f7a52bd0dc6b270082 in term 1.
I20260925 22:09:33.386915 27600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98fd2a0c80e546dda0f72772edd2a4ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a0f5f7b398f47f7a52bd0dc6b270082. Latest consensus state: current_term: 1 leader_uuid: "1a0f5f7b398f47f7a52bd0dc6b270082" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f5f7b398f47f7a52bd0dc6b270082" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 35921 } } peers { permanent_uuid: "98fd2a0c80e546dda0f72772edd2a4ba" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 45031 } } peers { permanent_uuid: "9c942fff072545a99aea4711922d5202" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 44541 } } }
I20260925 22:09:33.387171 27600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98fd2a0c80e546dda0f72772edd2a4ba [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:09:33.387212 27600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98fd2a0c80e546dda0f72772edd2a4ba [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1a0f5f7b398f47f7a52bd0dc6b270082" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f5f7b398f47f7a52bd0dc6b270082" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 35921 } } peers { permanent_uuid: "98fd2a0c80e546dda0f72772edd2a4ba" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 45031 } } peers { permanent_uuid: "9c942fff072545a99aea4711922d5202" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 44541 } } }
I20260925 22:09:33.387262 27600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98fd2a0c80e546dda0f72772edd2a4ba [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:09:33.388844 27615 catalog_manager.cc:1259] Loaded cluster ID: 356ab715ca4d4fafa5fc2c255c90e3b5
I20260925 22:09:33.388885 27615 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 98fd2a0c80e546dda0f72772edd2a4ba: loading cluster ID for follower catalog manager: success
W20260925 22:09:33.388971 27615 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 98fd2a0c80e546dda0f72772edd2a4ba: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260925 22:09:33.389114 27615 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:09:33.391449 27584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a0f5f7b398f47f7a52bd0dc6b270082 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1a0f5f7b398f47f7a52bd0dc6b270082" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f5f7b398f47f7a52bd0dc6b270082" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 35921 } } peers { permanent_uuid: "98fd2a0c80e546dda0f72772edd2a4ba" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 45031 } } peers { permanent_uuid: "9c942fff072545a99aea4711922d5202" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 44541 } } }
I20260925 22:09:33.391577 27584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a0f5f7b398f47f7a52bd0dc6b270082 [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:33.392508 27586 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:09:33.393208 27586 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:09:33.400535 27586 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a0f5f7b398f47f7a52bd0dc6b270082: Generated new TSK 0
I20260925 22:09:33.400704 27586 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 22:09:33.427476 27555 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:09:33.426094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38638 (local address 127.5.0.124:44541)
0925 22:09:33.426138 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:33.426140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:33.426821 (+ 681us) server_negotiation.cc:407] Connection header received
0925 22:09:33.426924 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:33.426925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:33.426936 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:33.426966 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:33.427395 (+ 429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38638: BlockingRecv error: recv got EOF from 127.0.0.1:38638 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260925 22:09:33.428205 27405 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:09:33.426516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47004 (local address 127.5.0.126:35921)
0925 22:09:33.426557 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:33.426559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:33.427706 (+ 1147us) server_negotiation.cc:407] Connection header received
0925 22:09:33.427799 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:33.427799 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:33.427809 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:33.427838 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:33.428147 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47004: BlockingRecv error: recv got EOF from 127.0.0.1:47004 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260925 22:09:33.429219 27480 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:09:33.428548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36292 (local address 127.5.0.125:45031)
0925 22:09:33.428587 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:33.428588 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:33.428711 (+ 123us) server_negotiation.cc:407] Connection header received
0925 22:09:33.428796 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:33.428797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:33.428806 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:33.428835 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:33.429156 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36292: BlockingRecv error: recv got EOF from 127.0.0.1:36292 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260925 22:09:33.431376 27480 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:09:33.430710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36302 (local address 127.5.0.125:45031)
0925 22:09:33.430742 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:33.430744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:33.430892 (+ 148us) server_negotiation.cc:407] Connection header received
0925 22:09:33.430989 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:33.430990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:33.431006 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:33.431035 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:33.431336 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36302: BlockingRecv error: recv got EOF from 127.0.0.1:36302 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260925 22:09:33.434264 27480 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:09:33.433597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36308 (local address 127.5.0.125:45031)
0925 22:09:33.433627 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:33.433628 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:33.433818 (+ 190us) server_negotiation.cc:407] Connection header received
0925 22:09:33.433905 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:33.433906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:33.433915 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:33.433942 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:33.434227 (+ 285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36308: BlockingRecv error: recv got EOF from 127.0.0.1:36308 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260925 22:09:33.437999 27480 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:09:33.437312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36316 (local address 127.5.0.125:45031)
0925 22:09:33.437355 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:33.437357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:33.437559 (+ 202us) server_negotiation.cc:407] Connection header received
0925 22:09:33.437646 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:33.437647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:33.437656 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:33.437683 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:33.437960 (+ 277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36316: BlockingRecv error: recv got EOF from 127.0.0.1:36316 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260925 22:09:33.444937 27480 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:09:33.444117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36330 (local address 127.5.0.125:45031)
0925 22:09:33.444150 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:33.444152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:33.444396 (+ 244us) server_negotiation.cc:407] Connection header received
0925 22:09:33.444548 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:33.444549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:33.444560 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:33.444588 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:33.444893 (+ 305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36330: BlockingRecv error: recv got EOF from 127.0.0.1:36330 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260925 22:09:33.455011 27480 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:09:33.454179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36338 (local address 127.5.0.125:45031)
0925 22:09:33.454216 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:33.454218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:33.454441 (+ 223us) server_negotiation.cc:407] Connection header received
0925 22:09:33.454553 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:33.454554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:33.454565 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:33.454595 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:33.454947 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36338: BlockingRecv error: recv got EOF from 127.0.0.1:36338 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260925 22:09:33.473477 27480 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:09:33.472454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36352 (local address 127.5.0.125:45031)
0925 22:09:33.472528 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:33.472530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:33.472817 (+ 287us) server_negotiation.cc:407] Connection header received
0925 22:09:33.472935 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:33.472936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:33.472949 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:33.472982 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:33.473409 (+ 427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36352: BlockingRecv error: recv got EOF from 127.0.0.1:36352 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260925 22:09:33.507872 27480 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:09:33.506937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36360 (local address 127.5.0.125:45031)
0925 22:09:33.506984 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:33.506987 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:33.507253 (+ 266us) server_negotiation.cc:407] Connection header received
0925 22:09:33.507359 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:33.507360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:33.507372 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:33.507403 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:33.507812 (+ 409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36360: BlockingRecv error: recv got EOF from 127.0.0.1:36360 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260925 22:09:33.574077 27480 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:09:33.573279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36374 (local address 127.5.0.125:45031)
0925 22:09:33.573351 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:33.573353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:33.573499 (+ 146us) server_negotiation.cc:407] Connection header received
0925 22:09:33.573587 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:33.573588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:33.573600 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:33.573670 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:33.574018 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36374: BlockingRecv error: recv got EOF from 127.0.0.1:36374 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260925 22:09:33.704192 27480 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:09:33.703394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36376 (local address 127.5.0.125:45031)
0925 22:09:33.703472 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:33.703475 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:33.703642 (+ 167us) server_negotiation.cc:407] Connection header received
0925 22:09:33.703689 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:33.703690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:33.703701 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:33.703757 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:33.704103 (+ 346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36376: BlockingRecv error: recv got EOF from 127.0.0.1:36376 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260925 22:09:33.962170 27480 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:09:33.961421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36384 (local address 127.5.0.125:45031)
0925 22:09:33.961496 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:33.961498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:33.961644 (+ 146us) server_negotiation.cc:407] Connection header received
0925 22:09:33.961676 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:33.961677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:33.961688 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:33.961733 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:33.962096 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36384: BlockingRecv error: recv got EOF from 127.0.0.1:36384 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 22:09:34.325246 27568 catalog_manager.cc:1259] Loaded cluster ID: 356ab715ca4d4fafa5fc2c255c90e3b5
I20260925 22:09:34.325325 27568 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9c942fff072545a99aea4711922d5202: loading cluster ID for follower catalog manager: success
I20260925 22:09:34.326185 27568 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9c942fff072545a99aea4711922d5202: acquiring CA information for follower catalog manager: success
I20260925 22:09:34.326555 27568 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9c942fff072545a99aea4711922d5202: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 22:09:34.390213 27615 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 98fd2a0c80e546dda0f72772edd2a4ba: acquiring CA information for follower catalog manager: success
I20260925 22:09:34.390594 27615 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 98fd2a0c80e546dda0f72772edd2a4ba: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 22:09:34.478101 27681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:34.475319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36396 (local address 127.5.0.125:45031)
0925 22:09:34.475450 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:34.475452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:34.475511 (+ 59us) server_negotiation.cc:407] Connection header received
0925 22:09:34.475624 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:34.475625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:34.475694 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:34.475787 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:09:34.475962 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:34.476493 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:34.477267 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:34.477475 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:34.477555 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:34.477744 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:09:34.477852 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:09:34.477942 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:34.477964 (+ 22us) server_negotiation.cc:299] Negotiation successful
0925 22:09:34.477986 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":59,"threads_started":1}
I20260925 22:09:34.482383 27687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:34.480086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38650 (local address 127.5.0.124:44541)
0925 22:09:34.480187 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:34.480188 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:34.480195 (+ 7us) server_negotiation.cc:407] Connection header received
0925 22:09:34.480229 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:34.480230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:34.480264 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:34.480333 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:09:34.480431 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:34.480902 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:34.481578 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:34.481759 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:34.481807 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:34.482057 (+ 250us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:09:34.482152 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:09:34.482204 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:34.482247 (+ 43us) server_negotiation.cc:299] Negotiation successful
0925 22:09:34.482264 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":56,"threads_started":1}
I20260925 22:09:34.487032 27693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:34.484046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47020 (local address 127.5.0.126:35921)
0925 22:09:34.484164 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:34.484166 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:34.484236 (+ 70us) server_negotiation.cc:407] Connection header received
0925 22:09:34.484321 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:34.484322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:34.484375 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:34.484452 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:09:34.484627 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:34.485350 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:34.486107 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:34.486419 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:34.486486 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:34.486688 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:09:34.486808 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:09:34.486868 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:34.486915 (+ 47us) server_negotiation.cc:299] Negotiation successful
0925 22:09:34.486937 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":65,"threads_started":1}
I20260925 22:09:34.487074 27681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:34.484046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36402 (local address 127.5.0.125:45031)
0925 22:09:34.484099 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:34.484102 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:34.484323 (+ 221us) server_negotiation.cc:407] Connection header received
0925 22:09:34.484437 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:34.484438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:34.484449 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:34.484497 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:09:34.484629 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:34.485084 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:34.486173 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:34.486475 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:34.486602 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:34.486765 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:09:34.486798 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:09:34.486841 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:34.486940 (+ 99us) server_negotiation.cc:299] Negotiation successful
0925 22:09:34.486966 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 22:09:34.487185 27687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:34.484121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38662 (local address 127.5.0.124:44541)
0925 22:09:34.484256 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:34.484258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:34.484359 (+ 101us) server_negotiation.cc:407] Connection header received
0925 22:09:34.484416 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:34.484417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:34.484427 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:34.484474 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:09:34.484569 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:34.485291 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:34.486314 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:34.486585 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:34.486642 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:34.486810 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:09:34.486843 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:09:34.486893 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:34.487092 (+ 199us) server_negotiation.cc:299] Negotiation successful
0925 22:09:34.487118 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260925 22:09:34.488969 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 27333
I20260925 22:09:34.494212 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 27408
I20260925 22:09:34.498914 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 27483
2026-09-25T22:09:34Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2772 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:09:34 dist-test-slave-7mqr krb5kdc[27698](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:09:34 dist-test-slave-7mqr krb5kdc[27698](info): set up 2 sockets
Sep 25 22:09:34 dist-test-slave-7mqr krb5kdc[27698](info): commencing operation
krb5kdc: starting...
W20260925 22:09:35.532112 5121 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/krb5kdc/test-user.keytab.
Sep 25 22:09:35 dist-test-slave-7mqr krb5kdc[27698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:09:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:09:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.0.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
Entry for principal kudu/127.5.0.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab.
I20260925 22:09:35.650450 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:42637
--webserver_interface=127.5.0.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:42637,127.5.0.125:44047,127.5.0.124:41289
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:09:35.731300 27711 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 22:09:35.731554 27711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:35.731583 27711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:35.731644 27711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:35.732858 27711 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:09:35.732929 27711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:35.732955 27711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:35.732986 27711 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:09:35.733014 27711 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:09:35.734496 27711 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:42637,127.5.0.125:44047,127.5.0.124:41289
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.126.keytab
--principal=kudu/127.5.0.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.126:42637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.0.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:35.734779 27711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:35.735008 27711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:35.739823 27719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:35.739799 27716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:35.739813 27717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:35.740095 27711 server_base.cc:1034] running on GCE node
I20260925 22:09:35.740346 27711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:35.740702 27711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:35.741873 27711 hybrid_clock.cc:648] HybridClock initialized: now 1790374175741860 us; error 28 us; skew 500 ppm
Sep 25 22:09:35 dist-test-slave-7mqr krb5kdc[27698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374175, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:35.744329 27711 init.cc:377] Logged in from keytab as kudu/127.5.0.126@KRBTEST.COM (short username kudu)
I20260925 22:09:35.744789 27711 webserver.cc:460] Webserver started at http://127.5.0.126:42895/ using document root <none> and password file <none>
I20260925 22:09:35.744973 27711 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:35.745013 27711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:35.745098 27711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:35.746088 27711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-0/data/instance:
uuid: "ecab3c9182a1424081dcfb3ad0ed056d"
format_stamp: "Formatted at 2026-09-25 22:09:35 on dist-test-slave-7mqr"
server_key: "bab08d4bbde0e46cfe439c3daf7c75f3"
server_key_iv: "d23fd92c4b0b373a20f1e6432b9f2d6e"
server_key_version: "encryptionkey@0"
I20260925 22:09:35.746395 27711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-0/wal/instance:
uuid: "ecab3c9182a1424081dcfb3ad0ed056d"
format_stamp: "Formatted at 2026-09-25 22:09:35 on dist-test-slave-7mqr"
server_key: "bab08d4bbde0e46cfe439c3daf7c75f3"
server_key_iv: "d23fd92c4b0b373a20f1e6432b9f2d6e"
server_key_version: "encryptionkey@0"
I20260925 22:09:35.747627 27711 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 22:09:35.748376 27726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:35.748528 27711 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:35.748612 27711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-0/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-0/wal
uuid: "ecab3c9182a1424081dcfb3ad0ed056d"
format_stamp: "Formatted at 2026-09-25 22:09:35 on dist-test-slave-7mqr"
server_key: "bab08d4bbde0e46cfe439c3daf7c75f3"
server_key_iv: "d23fd92c4b0b373a20f1e6432b9f2d6e"
server_key_version: "encryptionkey@0"
I20260925 22:09:35.748672 27711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:35.780787 27711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:35.783190 27711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:35.783301 27711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:35.787201 27711 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.126:42637
I20260925 22:09:35.787262 27778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.126:42637 every 8 connection(s)
I20260925 22:09:35.787604 27711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-0/data/info.pb
I20260925 22:09:35.788643 27779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:35.790282 27779 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42637 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:09:35.795334 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 27711
I20260925 22:09:35.795456 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-0/wal/instance
I20260925 22:09:35.795640 5121 external_mini_cluster.cc:1428] Setting key 909aa76197cace46d469b61785565fd9
Sep 25 22:09:35 dist-test-slave-7mqr krb5kdc[27698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374175, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:09:35.799873 27781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:35.790948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42637 (local address 127.0.0.1:35926)
0925 22:09:35.791111 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:35.791164 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:35.791602 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:35.791930 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:35.791932 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:35.792207 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:35.792352 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:35.792357 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:35.793387 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:35.793389 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:35.794056 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:35.794058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:35.794579 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:35.794581 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:35.794644 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:35.795151 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:09:35.795159 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:35.796604 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:35.798363 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:35.798369 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:35.798371 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:35.798566 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:35.798810 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:35.798811 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:35.798811 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:35.798899 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:35.799263 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:35.799267 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:35.799329 (+ 62us) client_negotiation.cc:769] Sending connection context
0925 22:09:35.799430 (+ 101us) client_negotiation.cc:240] Negotiation successful
0925 22:09:35.799671 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":99,"thread_start_us":102,"threads_started":1}
I20260925 22:09:35.799876 27782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:35.791056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35926 (local address 127.5.0.126:42637)
0925 22:09:35.791225 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:35.791265 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:35.791303 (+ 38us) server_negotiation.cc:407] Connection header received
0925 22:09:35.791784 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:35.791786 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:35.791811 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:35.791918 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:35.792457 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:35.793273 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:35.794140 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:35.794455 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:35.794585 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:35.796690 (+ 2105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:35.796696 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:35.796697 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:35.796703 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:35.798258 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:35.798655 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:35.798658 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:35.798658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:35.798741 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:35.798965 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:35.798966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:35.798966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:35.799181 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:35.799271 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:35.799468 (+ 197us) server_negotiation.cc:299] Negotiation successful
0925 22:09:35.799606 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":117,"threads_started":1}
I20260925 22:09:35.800773 27779 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.0.125" port: 44047 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:09:35.801134 27781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:35.801023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:44047 (local address 127.0.0.1:50956)
0925 22:09:35.801073 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:35.801114 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.125:44047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260925 22:09:35.801293 27730 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.0.125:44047: connect: Connection refused (error 111)
I20260925 22:09:35.801723 27781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:35.801632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:44047 (local address 127.0.0.1:50960)
0925 22:09:35.801681 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:35.801706 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.125:44047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260925 22:09:35.801841 27779 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.0.125:44047: Network error: Client connection negotiation failed: client connection to 127.5.0.125:44047: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.5.0.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.125.keytab.
Entry for principal kudu/127.5.0.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.125.keytab.
I20260925 22:09:35.818605 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.125:44047
--webserver_interface=127.5.0.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.125.keytab
--principal=kudu/127.5.0.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:42637,127.5.0.125:44047,127.5.0.124:41289
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:09:35.851042 27779 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.0.125" port: 44047 } attempt: 1
I20260925 22:09:35.851506 27781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:35.851374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:44047 (local address 127.0.0.1:50966)
0925 22:09:35.851437 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:35.851476 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.125:44047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260925 22:09:35.852128 27781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:35.852009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:44047 (local address 127.0.0.1:50976)
0925 22:09:35.852066 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:35.852098 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.125:44047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260925 22:09:35.852224 27779 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.0.125:44047: Network error: Client connection negotiation failed: client connection to 127.5.0.125:44047: connect: Connection refused (error 111)
W20260925 22:09:35.900763 27786 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 22:09:35.901012 27786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:35.901060 27786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:35.901160 27786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:35.902307 27786 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:09:35.902374 27786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:35.902406 27786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:35.902436 27786 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:09:35.902462 27786 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:09:35.904013 27786 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:42637,127.5.0.125:44047,127.5.0.124:41289
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.125.keytab
--principal=kudu/127.5.0.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.125:44047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.0.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27786
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:35.904289 27786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:35.904587 27786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:35.909242 27792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:35.909243 27791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:35.909366 27786 server_base.cc:1034] running on GCE node
W20260925 22:09:35.909271 27794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:35.909708 27786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:35.909996 27786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:35.911162 27786 hybrid_clock.cc:648] HybridClock initialized: now 1790374175911145 us; error 39 us; skew 500 ppm
Sep 25 22:09:35 dist-test-slave-7mqr krb5kdc[27698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374175, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:35.913668 27786 init.cc:377] Logged in from keytab as kudu/127.5.0.125@KRBTEST.COM (short username kudu)
I20260925 22:09:35.914134 27786 webserver.cc:460] Webserver started at http://127.5.0.125:32945/ using document root <none> and password file <none>
I20260925 22:09:35.914364 27786 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:35.914438 27786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:35.914589 27786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:35.915640 27786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-1/data/instance:
uuid: "9da53235e89c4810b4316ed69b319e5c"
format_stamp: "Formatted at 2026-09-25 22:09:35 on dist-test-slave-7mqr"
server_key: "5b77ff4109a888d043d392634e755b25"
server_key_iv: "6a5f042794a0fc737667bfb4e6ad10d3"
server_key_version: "encryptionkey@0"
I20260925 22:09:35.916136 27786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-1/wal/instance:
uuid: "9da53235e89c4810b4316ed69b319e5c"
format_stamp: "Formatted at 2026-09-25 22:09:35 on dist-test-slave-7mqr"
server_key: "5b77ff4109a888d043d392634e755b25"
server_key_iv: "6a5f042794a0fc737667bfb4e6ad10d3"
server_key_version: "encryptionkey@0"
I20260925 22:09:35.917472 27786 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:09:35.918241 27801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:35.918418 27786 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:09:35.918524 27786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-1/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-1/wal
uuid: "9da53235e89c4810b4316ed69b319e5c"
format_stamp: "Formatted at 2026-09-25 22:09:35 on dist-test-slave-7mqr"
server_key: "5b77ff4109a888d043d392634e755b25"
server_key_iv: "6a5f042794a0fc737667bfb4e6ad10d3"
server_key_version: "encryptionkey@0"
I20260925 22:09:35.918610 27786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:35.920356 27779 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.0.125" port: 44047 } attempt: 2
I20260925 22:09:35.920717 27781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:35.920599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:44047 (local address 127.0.0.1:50980)
0925 22:09:35.920656 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:35.920689 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.125:44047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260925 22:09:35.921031 27781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:35.920936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:44047 (local address 127.0.0.1:50984)
0925 22:09:35.920982 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:35.921012 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.125:44047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260925 22:09:35.921123 27779 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.0.125:44047: Network error: Client connection negotiation failed: client connection to 127.5.0.125:44047: connect: Connection refused (error 111)
I20260925 22:09:35.950757 27786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:35.952837 27786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:35.953014 27786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:35.957015 27786 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.125:44047
I20260925 22:09:35.957065 27853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.125:44047 every 8 connection(s)
I20260925 22:09:35.957464 27786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-1/data/info.pb
I20260925 22:09:35.958410 27854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:35.959985 27854 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42637 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:09:35.963583 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 27786
I20260925 22:09:35.963687 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-1/wal/instance
I20260925 22:09:35.963878 5121 external_mini_cluster.cc:1428] Setting key 715dd56b2382a2fa69f9b849645f710f
Sep 25 22:09:35 dist-test-slave-7mqr krb5kdc[27698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374175, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.125@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:09:35.969714 27782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:35.960695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35942 (local address 127.5.0.126:42637)
0925 22:09:35.960752 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:35.960754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:35.960973 (+ 219us) server_negotiation.cc:407] Connection header received
0925 22:09:35.961486 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:35.961487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:35.961499 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:35.961563 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:35.962185 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:35.962717 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:35.964020 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:35.964236 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:35.964299 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:35.966796 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:35.966798 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:35.966798 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:35.966805 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:35.968328 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:35.968745 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:35.968746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:35.968746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:35.968809 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:35.969077 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:35.969078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:35.969078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:35.969192 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:35.969268 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:35.969534 (+ 266us) server_negotiation.cc:299] Negotiation successful
0925 22:09:35.969624 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 22:09:35.969894 27856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:35.960678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42637 (local address 127.0.0.1:35942)
0925 22:09:35.960842 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:35.960878 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:35.961367 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:35.961639 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:35.961642 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:35.961896 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:35.962089 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:35.962093 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:35.962834 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:35.962835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:35.963926 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:35.963929 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:35.964342 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:35.964343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:35.964508 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:35.964975 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:09:35.964985 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:35.966697 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:35.968430 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:35.968437 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:35.968439 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:35.968664 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:35.968878 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:35.968879 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:35.968880 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:35.968991 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:35.969261 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:35.969267 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:35.969405 (+ 138us) client_negotiation.cc:769] Sending connection context
0925 22:09:35.969513 (+ 108us) client_negotiation.cc:240] Negotiation successful
0925 22:09:35.969663 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":77,"threads_started":1}
I20260925 22:09:35.970407 27854 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.0.125" port: 44047 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.5.0.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.0.124@KRBTEST.COM" created.
Sep 25 22:09:35 dist-test-slave-7mqr krb5kdc[27698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374175, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.125@KRBTEST.COM for kudu/127.5.0.125@KRBTEST.COM
I20260925 22:09:35.977644 27856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:35.970692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:44047 (local address 127.0.0.1:50986)
0925 22:09:35.970864 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:35.970875 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:35.970922 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:35.971061 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:35.971061 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:35.971199 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:35.971238 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:35.971240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:35.972218 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:35.972219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:35.972745 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:35.972747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:35.973234 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:35.973235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:35.973336 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:35.973720 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:09:35.973725 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:35.974881 (+ 1156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:35.976614 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:35.976615 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:35.976616 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:35.976777 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:35.976984 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:35.976985 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:35.976986 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:35.977038 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:35.977324 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:35.977325 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:35.977363 (+ 38us) client_negotiation.cc:769] Sending connection context
0925 22:09:35.977428 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 22:09:35.977526 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160}
I20260925 22:09:35.977705 27858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:35.970725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50986 (local address 127.5.0.125:44047)
0925 22:09:35.970877 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:35.970882 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:35.970921 (+ 39us) server_negotiation.cc:407] Connection header received
0925 22:09:35.970978 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:35.970980 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:35.970992 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:35.971061 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:35.971320 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:35.972127 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:35.972839 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:35.973152 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:35.973238 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:35.974977 (+ 1739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:35.974985 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:35.974985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:35.974991 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:35.976508 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:35.976864 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:35.976867 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:35.976867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:35.976920 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:35.977111 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:35.977112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:35.977113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:35.977244 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:35.977332 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:35.977469 (+ 137us) server_negotiation.cc:299] Negotiation successful
0925 22:09:35.977579 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":123,"thread_start_us":96,"threads_started":1}
I20260925 22:09:35.978236 27854 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.0.124" port: 41289 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:09:35.978571 27856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:35.978455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:41289 (local address 127.0.0.1:41608)
0925 22:09:35.978501 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:35.978547 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.124:41289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260925 22:09:35.978708 27802 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.0.124:41289: connect: Connection refused (error 111)
I20260925 22:09:35.979132 27856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:35.979038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:41289 (local address 127.0.0.1:41614)
0925 22:09:35.979087 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:35.979110 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.124:41289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260925 22:09:35.979234 27854 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.0.124:41289: Network error: Client connection negotiation failed: client connection to 127.5.0.124:41289: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.0.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.124.keytab.
Entry for principal kudu/127.5.0.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.124.keytab.
I20260925 22:09:35.988284 5121 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMOkht1/build/release/bin/kudu
/tmp/dist-test-taskMOkht1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.124:41289
--webserver_interface=127.5.0.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.124.keytab
--principal=kudu/127.5.0.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.0.84:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.0.126:42637,127.5.0.125:44047,127.5.0.124:41289
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:09:36.012269 27779 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.0.125" port: 44047 } attempt: 3
Sep 25 22:09:36 dist-test-slave-7mqr krb5kdc[27698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374175, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for kudu/127.5.0.125@KRBTEST.COM
I20260925 22:09:36.019838 27781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:36.012698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:44047 (local address 127.0.0.1:50988)
0925 22:09:36.012776 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:36.012788 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:36.012851 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:36.012997 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:36.012998 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:36.013153 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:36.013217 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:36.013218 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.014136 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.014136 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:36.014643 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:36.014644 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.015218 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.015218 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:36.015274 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:36.015758 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:09:36.015763 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:36.017207 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:36.018954 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:36.018955 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:36.018955 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:36.019132 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:36.019323 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:36.019323 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:36.019324 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:36.019372 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:36.019615 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:36.019615 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:36.019652 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 22:09:36.019701 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 22:09:36.019770 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260925 22:09:36.019914 27858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:36.012759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50988 (local address 127.5.0.125:44047)
0925 22:09:36.012831 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:36.012834 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:36.012846 (+ 12us) server_negotiation.cc:407] Connection header received
0925 22:09:36.012902 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:36.012903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:36.012916 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:36.012983 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:36.013288 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.014052 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.014798 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.015125 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.015205 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:36.017288 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:36.017289 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:36.017289 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:36.017297 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:36.018864 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:36.019194 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:36.019195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:36.019195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:36.019258 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:36.019427 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:36.019428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:36.019428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:36.019556 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:36.019621 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:36.019725 (+ 104us) server_negotiation.cc:299] Negotiation successful
0925 22:09:36.019802 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 22:09:36.020453 27779 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.0.124" port: 41289 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:09:36.020794 27781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:36.020696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:41289 (local address 127.0.0.1:41616)
0925 22:09:36.020742 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:36.020772 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.124:41289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260925 22:09:36.021090 27781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:36.021025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:41289 (local address 127.0.0.1:41628)
0925 22:09:36.021045 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:36.021076 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.124:41289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260925 22:09:36.021201 27779 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.0.124:41289: Network error: Client connection negotiation failed: client connection to 127.5.0.124:41289: connect: Connection refused (error 111)
I20260925 22:09:36.028467 27854 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.0.124" port: 41289 } attempt: 1
I20260925 22:09:36.028855 27856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:36.028747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:41289 (local address 127.0.0.1:41632)
0925 22:09:36.028800 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:36.028832 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.124:41289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260925 22:09:36.029232 27856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:36.029155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:41289 (local address 127.0.0.1:41638)
0925 22:09:36.029192 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:36.029217 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.124:41289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260925 22:09:36.029353 27854 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.0.124:41289: Network error: Client connection negotiation failed: client connection to 127.5.0.124:41289: connect: Connection refused (error 111)
I20260925 22:09:36.052356 27779 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.0.124" port: 41289 } attempt: 1
I20260925 22:09:36.052824 27781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:36.052685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:41289 (local address 127.0.0.1:41642)
0925 22:09:36.052745 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:36.052783 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.124:41289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260925 22:09:36.053170 27781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:36.053073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:41289 (local address 127.0.0.1:41652)
0925 22:09:36.053120 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:36.053145 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.124:41289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260925 22:09:36.053280 27779 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.0.124:41289: Network error: Client connection negotiation failed: client connection to 127.5.0.124:41289: connect: Connection refused (error 111)
W20260925 22:09:36.073930 27861 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 22:09:36.074131 27861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:09:36.074147 27861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:09:36.074167 27861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:09:36.075280 27861 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:09:36.075320 27861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:09:36.075333 27861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:09:36.075346 27861 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:09:36.075397 27861 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:09:36.076905 27861 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.0.84:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.0.126:42637,127.5.0.125:44047,127.5.0.124:41289
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/krb5kdc/kudu_127.5.0.124.keytab
--principal=kudu/127.5.0.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.0.124:41289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.0.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:09:36.077149 27861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:09:36.077384 27861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:09:36.082057 27867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:36.082085 27866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:09:36.082047 27869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:09:36.082342 27861 server_base.cc:1034] running on GCE node
I20260925 22:09:36.082573 27861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:09:36.082798 27861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:09:36.083954 27861 hybrid_clock.cc:648] HybridClock initialized: now 1790374176083931 us; error 31 us; skew 500 ppm
Sep 25 22:09:36 dist-test-slave-7mqr krb5kdc[27698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374176, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:09:36.086288 27861 init.cc:377] Logged in from keytab as kudu/127.5.0.124@KRBTEST.COM (short username kudu)
I20260925 22:09:36.086723 27861 webserver.cc:460] Webserver started at http://127.5.0.124:42931/ using document root <none> and password file <none>
I20260925 22:09:36.086910 27861 fs_manager.cc:359] Metadata directory not provided
I20260925 22:09:36.086949 27861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:09:36.087033 27861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:09:36.088069 27861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-2/data/instance:
uuid: "c9b34a9d8c254e60a6f02e8d38cb6bb8"
format_stamp: "Formatted at 2026-09-25 22:09:36 on dist-test-slave-7mqr"
server_key: "cb7de95550933ebf6b8fb5b98120ea36"
server_key_iv: "7ad6c5c1d910b1a78b922116e29d27a1"
server_key_version: "encryptionkey@0"
I20260925 22:09:36.088357 27861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-2/wal/instance:
uuid: "c9b34a9d8c254e60a6f02e8d38cb6bb8"
format_stamp: "Formatted at 2026-09-25 22:09:36 on dist-test-slave-7mqr"
server_key: "cb7de95550933ebf6b8fb5b98120ea36"
server_key_iv: "7ad6c5c1d910b1a78b922116e29d27a1"
server_key_version: "encryptionkey@0"
I20260925 22:09:36.089628 27861 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:09:36.090309 27876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:09:36.090488 27861 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:09:36.090602 27861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-2/data,/tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-2/wal
uuid: "c9b34a9d8c254e60a6f02e8d38cb6bb8"
format_stamp: "Formatted at 2026-09-25 22:09:36 on dist-test-slave-7mqr"
server_key: "cb7de95550933ebf6b8fb5b98120ea36"
server_key_iv: "7ad6c5c1d910b1a78b922116e29d27a1"
server_key_version: "encryptionkey@0"
I20260925 22:09:36.090678 27861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:09:36.097489 27854 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.0.124" port: 41289 } attempt: 2
I20260925 22:09:36.097867 27856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:36.097730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:41289 (local address 127.0.0.1:41654)
0925 22:09:36.097791 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:36.097831 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.124:41289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260925 22:09:36.098219 27856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:36.098127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:41289 (local address 127.0.0.1:41660)
0925 22:09:36.098172 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:36.098198 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.0.124:41289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260925 22:09:36.098335 27854 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.0.124:41289: Network error: Client connection negotiation failed: client connection to 127.5.0.124:41289: connect: Connection refused (error 111)
I20260925 22:09:36.102344 27861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:09:36.104138 27861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:09:36.104244 27861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:09:36.108078 27861 rpc_server.cc:307] RPC server started. Bound to: 127.5.0.124:41289
I20260925 22:09:36.108122 27928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.0.124:41289 every 8 connection(s)
I20260925 22:09:36.108542 27861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-2/data/info.pb
I20260925 22:09:36.109632 27929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:09:36.111122 27929 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42637 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:09:36.112936 5121 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMOkht1/build/release/bin/kudu as pid 27861
I20260925 22:09:36.113052 5121 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMOkht1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373908987421-5121-0/minicluster-data/master-2/wal/instance
I20260925 22:09:36.113281 5121 external_mini_cluster.cc:1428] Setting key e157c37f7ab9149541a59f93ab0ac01c
Sep 25 22:09:36 dist-test-slave-7mqr krb5kdc[27698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374176, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.124@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
Sep 25 22:09:36 dist-test-slave-7mqr krb5kdc[27698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.124@KRBTEST.COM
I20260925 22:09:36.121073 27782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:36.111805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35958 (local address 127.5.0.126:42637)
0925 22:09:36.111863 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:36.111865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:36.112071 (+ 206us) server_negotiation.cc:407] Connection header received
0925 22:09:36.112507 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:36.112508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:36.112517 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:36.112577 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:36.113426 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.114040 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.115080 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.115284 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.115372 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:36.117500 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:36.117502 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:36.117503 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:36.117512 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:36.119596 (+ 2084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:36.120088 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:36.120090 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:36.120090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:36.120157 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:36.120451 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:36.120452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:36.120452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:36.120559 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:36.120607 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:36.120932 (+ 325us) server_negotiation.cc:299] Negotiation successful
0925 22:09:36.120993 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 22:09:36.121306 27931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:36.111751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.126:42637 (local address 127.0.0.1:35958)
0925 22:09:36.111923 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:36.111964 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:36.112430 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:36.112644 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:36.112647 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:36.113041 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:36.113334 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:36.113341 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.114145 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.114146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:36.114986 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:36.114988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.115400 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.115402 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:36.115519 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:36.116077 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:09:36.116085 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:36.117406 (+ 1321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:36.119671 (+ 2265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:36.119678 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:36.119680 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:36.120014 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:36.120230 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:36.120230 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:36.120231 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:36.120373 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:36.120625 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:36.120629 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:36.120777 (+ 148us) client_negotiation.cc:769] Sending connection context
0925 22:09:36.120887 (+ 110us) client_negotiation.cc:240] Negotiation successful
0925 22:09:36.121080 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
I20260925 22:09:36.121874 27929 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.0.125" port: 44047 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:09:36.122470 27933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:36.113638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41670 (local address 127.5.0.124:41289)
0925 22:09:36.113835 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:36.113841 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:36.113882 (+ 41us) server_negotiation.cc:407] Connection header received
0925 22:09:36.113951 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:36.113954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:36.113973 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:36.114052 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:36.114633 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.115318 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.116158 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.116462 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.116534 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:36.119121 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:36.119131 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:36.119131 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:36.119142 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:36.121076 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:36.121598 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:36.121601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:36.121602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:36.121660 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:36.121859 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:36.121860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:36.121861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:36.122011 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:36.122162 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:36.122283 (+ 121us) server_negotiation.cc:299] Negotiation successful
0925 22:09:36.122358 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":109,"threads_started":1}
Sep 25 22:09:36 dist-test-slave-7mqr krb5kdc[27698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374176, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.124@KRBTEST.COM for kudu/127.5.0.125@KRBTEST.COM
Sep 25 22:09:36 dist-test-slave-7mqr krb5kdc[27698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.126@KRBTEST.COM
I20260925 22:09:36.128466 27779 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.0.124" port: 41289 } attempt: 2
I20260925 22:09:36.128714 27931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:36.122129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.125:44047 (local address 127.0.0.1:50996)
0925 22:09:36.122229 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:36.122238 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:36.122297 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:36.122431 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:36.122432 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:36.122583 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:36.122636 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:36.122638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.123459 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.123460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:36.124025 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:36.124027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.124418 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.124419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:36.124492 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:36.124877 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:09:36.124881 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:36.125976 (+ 1095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:36.127653 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:36.127654 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:36.127654 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:36.127819 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:36.128052 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:36.128053 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:36.128053 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:36.128118 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:36.128354 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:36.128355 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:36.128400 (+ 45us) client_negotiation.cc:769] Sending connection context
0925 22:09:36.128503 (+ 103us) client_negotiation.cc:240] Negotiation successful
0925 22:09:36.128597 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260925 22:09:36.128727 27858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:36.122181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50996 (local address 127.5.0.125:44047)
0925 22:09:36.122231 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:36.122233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:36.122287 (+ 54us) server_negotiation.cc:407] Connection header received
0925 22:09:36.122358 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:36.122360 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:36.122370 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:36.122430 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:36.122701 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.123379 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.124111 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.124332 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.124418 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:36.126053 (+ 1635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:36.126055 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:36.126055 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:36.126063 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:36.127527 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:36.127916 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:36.127917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:36.127917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:36.127976 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:36.128187 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:36.128187 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:36.128188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:36.128287 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:36.128339 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:36.128539 (+ 200us) server_negotiation.cc:299] Negotiation successful
0925 22:09:36.128625 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 22:09:36.129078 27929 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.0.124" port: 41289 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:09:36.131422 27782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:36.123189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35972 (local address 127.5.0.126:42637)
0925 22:09:36.123226 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:36.123228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:36.123235 (+ 7us) server_negotiation.cc:407] Connection header received
0925 22:09:36.123274 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:36.123275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:36.123284 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:36.123337 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:36.123796 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.124312 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.125135 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.125334 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.125405 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:36.128059 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:36.128060 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:36.128061 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:36.128067 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:36.129945 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:36.130339 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:36.130340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:36.130341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:36.130400 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:36.130619 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:36.130620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:36.130620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:36.130730 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:36.130863 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:36.131259 (+ 396us) server_negotiation.cc:299] Negotiation successful
0925 22:09:36.131334 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Sep 25 22:09:36 dist-test-slave-7mqr krb5kdc[27698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374175, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.126@KRBTEST.COM for kudu/127.5.0.124@KRBTEST.COM
Sep 25 22:09:36 dist-test-slave-7mqr krb5kdc[27698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374176, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.124@KRBTEST.COM for kudu/127.5.0.124@KRBTEST.COM
I20260925 22:09:36.135931 27781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:36.128703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:41289 (local address 127.0.0.1:41672)
0925 22:09:36.128750 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:36.128761 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:36.128795 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:36.129061 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:36.129062 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:36.129207 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:36.129238 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:36.129240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.130007 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.130008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:36.130810 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:36.130812 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.131271 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.131272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:36.131339 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:36.131803 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:09:36.131806 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:36.133226 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:36.134862 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:36.134863 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:36.134864 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:36.135079 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:36.135268 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:36.135269 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:36.135269 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:36.135322 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:36.135591 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:36.135592 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:36.135636 (+ 44us) client_negotiation.cc:769] Sending connection context
0925 22:09:36.135691 (+ 55us) client_negotiation.cc:240] Negotiation successful
0925 22:09:36.135793 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260925 22:09:36.136216 27933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:36.128761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41672 (local address 127.5.0.124:41289)
0925 22:09:36.128863 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:36.128865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:36.128873 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:09:36.128895 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:36.128896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:36.128908 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:36.128960 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:36.129338 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.129930 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.130883 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.131197 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.131269 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:36.133316 (+ 2047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:36.133318 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:36.133318 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:36.133326 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:36.134783 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:36.135152 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:36.135154 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:36.135154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:36.135210 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:36.135404 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:36.135405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:36.135405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:36.135515 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:36.136039 (+ 524us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:36.136084 (+ 45us) server_negotiation.cc:299] Negotiation successful
0925 22:09:36.136161 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260925 22:09:36.137944 27931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:36.129462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:41289 (local address 127.0.0.1:41678)
0925 22:09:36.129999 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:36.130035 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:36.130074 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:36.130199 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:36.130200 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:36.130343 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:36.130373 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:36.130375 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.131103 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.131104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:36.131860 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:36.131863 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.132398 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.132399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:36.132445 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:36.132836 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:09:36.132842 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:36.134096 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:36.136799 (+ 2703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:36.136799 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:36.136800 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:36.137017 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:36.137228 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:36.137229 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:36.137229 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:36.137290 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:36.137614 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:36.137615 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:36.137661 (+ 46us) client_negotiation.cc:769] Sending connection context
0925 22:09:36.137740 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 22:09:36.137835 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527}
I20260925 22:09:36.137957 27779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecab3c9182a1424081dcfb3ad0ed056d: Bootstrap starting.
Sep 25 22:09:36 dist-test-slave-7mqr krb5kdc[27698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.0.125@KRBTEST.COM
I20260925 22:09:36.138090 27934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:36.129526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41678 (local address 127.5.0.124:41289)
0925 22:09:36.129669 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:36.129671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:36.130068 (+ 397us) server_negotiation.cc:407] Connection header received
0925 22:09:36.130121 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:36.130121 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:36.130132 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:36.130196 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:36.130456 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.131041 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.131958 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.132294 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.132355 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:36.134180 (+ 1825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:36.134182 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:36.134182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:36.134189 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:36.136709 (+ 2520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:36.137097 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:36.137098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:36.137098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:36.137160 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:36.137360 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:36.137361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:36.137361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:36.137524 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:36.137604 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:36.137784 (+ 180us) server_negotiation.cc:299] Negotiation successful
0925 22:09:36.137890 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":122,"thread_start_us":92,"threads_started":1}
I20260925 22:09:36.138674 27779 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ecab3c9182a1424081dcfb3ad0ed056d: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:36.138947 27779 log.cc:824] T 00000000000000000000000000000000 P ecab3c9182a1424081dcfb3ad0ed056d: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:36.139415 27929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9b34a9d8c254e60a6f02e8d38cb6bb8: Bootstrap starting.
I20260925 22:09:36.140159 27779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecab3c9182a1424081dcfb3ad0ed056d: No bootstrap required, opened a new log
I20260925 22:09:36.140165 27929 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9b34a9d8c254e60a6f02e8d38cb6bb8: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:36.140460 27929 log.cc:824] T 00000000000000000000000000000000 P c9b34a9d8c254e60a6f02e8d38cb6bb8: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:36.141263 27929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9b34a9d8c254e60a6f02e8d38cb6bb8: No bootstrap required, opened a new log
I20260925 22:09:36.141804 27858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:36.132228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51000 (local address 127.5.0.125:44047)
0925 22:09:36.132260 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:36.132262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:36.132335 (+ 73us) server_negotiation.cc:407] Connection header received
0925 22:09:36.132396 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:36.132397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:36.132407 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:36.132464 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:36.133051 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.133783 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.135143 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.135448 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.135491 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:36.138912 (+ 3421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:36.138913 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:36.138914 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:36.138922 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:36.140675 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:36.141031 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:36.141032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:36.141032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:36.141093 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:36.141300 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:36.141301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:36.141301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:36.141426 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:36.141599 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:36.141636 (+ 37us) server_negotiation.cc:299] Negotiation successful
0925 22:09:36.141705 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 22:09:36.142258 27779 raft_consensus.cc:348] T 00000000000000000000000000000000 P ecab3c9182a1424081dcfb3ad0ed056d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecab3c9182a1424081dcfb3ad0ed056d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42637 } } peers { permanent_uuid: "9da53235e89c4810b4316ed69b319e5c" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 44047 } } peers { permanent_uuid: "c9b34a9d8c254e60a6f02e8d38cb6bb8" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 41289 } }
I20260925 22:09:36.142408 27779 raft_consensus.cc:729] T 00000000000000000000000000000000 P ecab3c9182a1424081dcfb3ad0ed056d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecab3c9182a1424081dcfb3ad0ed056d, State: Initialized, Role: FOLLOWER
I20260925 22:09:36.142490 27779 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecab3c9182a1424081dcfb3ad0ed056d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecab3c9182a1424081dcfb3ad0ed056d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42637 } } peers { permanent_uuid: "9da53235e89c4810b4316ed69b319e5c" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 44047 } } peers { permanent_uuid: "c9b34a9d8c254e60a6f02e8d38cb6bb8" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 41289 } }
I20260925 22:09:36.142848 27779 sys_catalog.cc:564] T 00000000000000000000000000000000 P ecab3c9182a1424081dcfb3ad0ed056d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:09:36.142867 27937 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecab3c9182a1424081dcfb3ad0ed056d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecab3c9182a1424081dcfb3ad0ed056d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42637 } } peers { permanent_uuid: "9da53235e89c4810b4316ed69b319e5c" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 44047 } } peers { permanent_uuid: "c9b34a9d8c254e60a6f02e8d38cb6bb8" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 41289 } } }
I20260925 22:09:36.143092 27937 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecab3c9182a1424081dcfb3ad0ed056d [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:09:36.143162 5121 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:09:36.143647 27929 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9b34a9d8c254e60a6f02e8d38cb6bb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecab3c9182a1424081dcfb3ad0ed056d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42637 } } peers { permanent_uuid: "9da53235e89c4810b4316ed69b319e5c" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 44047 } } peers { permanent_uuid: "c9b34a9d8c254e60a6f02e8d38cb6bb8" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 41289 } }
I20260925 22:09:36.143800 27929 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9b34a9d8c254e60a6f02e8d38cb6bb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9b34a9d8c254e60a6f02e8d38cb6bb8, State: Initialized, Role: FOLLOWER
I20260925 22:09:36.143878 27929 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9b34a9d8c254e60a6f02e8d38cb6bb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecab3c9182a1424081dcfb3ad0ed056d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42637 } } peers { permanent_uuid: "9da53235e89c4810b4316ed69b319e5c" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 44047 } } peers { permanent_uuid: "c9b34a9d8c254e60a6f02e8d38cb6bb8" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 41289 } }
I20260925 22:09:36.144042 27753 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ecab3c9182a1424081dcfb3ad0ed056d"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35972
I20260925 22:09:36.144169 27753 raft_consensus.cc:482] T 00000000000000000000000000000000 P ecab3c9182a1424081dcfb3ad0ed056d [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260925 22:09:36.144222 27929 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9b34a9d8c254e60a6f02e8d38cb6bb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:09:36.144230 27753 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ecab3c9182a1424081dcfb3ad0ed056d [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:36.144304 27946 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9b34a9d8c254e60a6f02e8d38cb6bb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecab3c9182a1424081dcfb3ad0ed056d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42637 } } peers { permanent_uuid: "9da53235e89c4810b4316ed69b319e5c" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 44047 } } peers { permanent_uuid: "c9b34a9d8c254e60a6f02e8d38cb6bb8" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 41289 } } }
I20260925 22:09:36.144411 27946 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9b34a9d8c254e60a6f02e8d38cb6bb8 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:09:36.145056 27753 raft_consensus.cc:504] T 00000000000000000000000000000000 P ecab3c9182a1424081dcfb3ad0ed056d [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecab3c9182a1424081dcfb3ad0ed056d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42637 } } peers { permanent_uuid: "9da53235e89c4810b4316ed69b319e5c" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 44047 } } peers { permanent_uuid: "c9b34a9d8c254e60a6f02e8d38cb6bb8" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 41289 } }
I20260925 22:09:36.145332 27753 leader_election.cc:288] T 00000000000000000000000000000000 P ecab3c9182a1424081dcfb3ad0ed056d [CANDIDATE]: Term 1 election: Requested vote from peers 9da53235e89c4810b4316ed69b319e5c (127.5.0.125:44047), c9b34a9d8c254e60a6f02e8d38cb6bb8 (127.5.0.124:41289)
I20260925 22:09:36.145776 27903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ecab3c9182a1424081dcfb3ad0ed056d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c9b34a9d8c254e60a6f02e8d38cb6bb8"
I20260925 22:09:36.146636 27903 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9b34a9d8c254e60a6f02e8d38cb6bb8 [term 0 FOLLOWER]: Advancing to term 1
W20260925 22:09:36.147215 27960 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ecab3c9182a1424081dcfb3ad0ed056d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:09:36.147310 27960 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:09:36.147770 27903 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c9b34a9d8c254e60a6f02e8d38cb6bb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecab3c9182a1424081dcfb3ad0ed056d in term 1.
I20260925 22:09:36.148038 27727 leader_election.cc:302] T 00000000000000000000000000000000 P ecab3c9182a1424081dcfb3ad0ed056d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c9b34a9d8c254e60a6f02e8d38cb6bb8, ecab3c9182a1424081dcfb3ad0ed056d; no voters:
W20260925 22:09:36.148307 27959 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c9b34a9d8c254e60a6f02e8d38cb6bb8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:09:36.148360 27959 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:09:36.147203 27711 master_runner.cc:186] Error getting master registration for 127.5.0.125:44047: OK, instance_id { permanent_uuid: "9da53235e89c4810b4316ed69b319e5c" instance_seqno: 1790374175955791 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 22:09:36.149981 27711 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 22:09:36.150056 27937 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ecab3c9182a1424081dcfb3ad0ed056d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:09:36.150211 27937 raft_consensus.cc:686] T 00000000000000000000000000000000 P ecab3c9182a1424081dcfb3ad0ed056d [term 1 LEADER]: Becoming Leader. State: Replica: ecab3c9182a1424081dcfb3ad0ed056d, State: Running, Role: LEADER
I20260925 22:09:36.150301 27937 consensus_queue.cc:237] T 00000000000000000000000000000000 P ecab3c9182a1424081dcfb3ad0ed056d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecab3c9182a1424081dcfb3ad0ed056d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42637 } } peers { permanent_uuid: "9da53235e89c4810b4316ed69b319e5c" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 44047 } } peers { permanent_uuid: "c9b34a9d8c254e60a6f02e8d38cb6bb8" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 41289 } }
I20260925 22:09:36.150739 27970 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecab3c9182a1424081dcfb3ad0ed056d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ecab3c9182a1424081dcfb3ad0ed056d. Latest consensus state: current_term: 1 leader_uuid: "ecab3c9182a1424081dcfb3ad0ed056d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecab3c9182a1424081dcfb3ad0ed056d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42637 } } peers { permanent_uuid: "9da53235e89c4810b4316ed69b319e5c" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 44047 } } peers { permanent_uuid: "c9b34a9d8c254e60a6f02e8d38cb6bb8" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 41289 } } }
I20260925 22:09:36.150895 27970 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecab3c9182a1424081dcfb3ad0ed056d [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:36.151679 27861 master_runner.cc:186] Error getting master registration for 127.5.0.125:44047: OK, instance_id { permanent_uuid: "9da53235e89c4810b4316ed69b319e5c" instance_seqno: 1790374175955791 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 22:09:36.153401 27861 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 22:09:36.155578 27972 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:09:36.155788 27972 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:09:36.158592 27934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:36.146610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41688 (local address 127.5.0.124:41289)
0925 22:09:36.146645 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:36.146647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:36.148736 (+ 2089us) server_negotiation.cc:407] Connection header received
0925 22:09:36.148805 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:36.148806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:36.148817 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:36.148869 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:36.149416 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.149905 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.150905 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.151199 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.151245 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:36.154505 (+ 3260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:36.154506 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:36.154507 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:36.154514 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:36.156748 (+ 2234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:36.157339 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:36.157341 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:36.157341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:36.157414 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:36.157773 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:36.157775 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:36.157789 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:36.157926 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:36.158017 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:36.158386 (+ 369us) server_negotiation.cc:299] Negotiation successful
0925 22:09:36.158465 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:09:36.160892 27858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:36.146469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51004 (local address 127.5.0.125:44047)
0925 22:09:36.146517 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:36.146520 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:36.152087 (+ 5567us) server_negotiation.cc:407] Connection header received
0925 22:09:36.152189 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:36.152190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:36.152200 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:36.152229 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:36.152958 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.153657 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.154648 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.154953 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.155018 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:36.157434 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:36.157435 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:36.157435 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:36.157442 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:36.159689 (+ 2247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:36.160071 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:36.160072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:36.160073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:36.160124 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:36.160357 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:36.160358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:36.160358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:36.160479 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:36.160583 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:36.160718 (+ 135us) server_negotiation.cc:299] Negotiation successful
0925 22:09:36.160792 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 22:09:36.167748 27782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:36.148371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35988 (local address 127.5.0.126:42637)
0925 22:09:36.148412 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:36.148415 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:36.154742 (+ 6327us) server_negotiation.cc:407] Connection header received
0925 22:09:36.154836 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:36.154837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:36.154847 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:36.154876 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:36.155479 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.156250 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.157203 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.157496 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.157556 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:36.164891 (+ 7335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:36.164892 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:36.164893 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:36.164900 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:36.166611 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:36.166969 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:36.166970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:36.166970 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:36.167029 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:36.167222 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:36.167223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:36.167223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:36.167362 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:36.167464 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:36.167575 (+ 111us) server_negotiation.cc:299] Negotiation successful
0925 22:09:36.167646 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 22:09:36.172859 27903 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c9b34a9d8c254e60a6f02e8d38cb6bb8 [term 1 FOLLOWER]: Refusing update from remote peer ecab3c9182a1424081dcfb3ad0ed056d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:09:36.173199 27970 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ecab3c9182a1424081dcfb3ad0ed056d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9b34a9d8c254e60a6f02e8d38cb6bb8" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:09:36.174582 27946 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9b34a9d8c254e60a6f02e8d38cb6bb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ecab3c9182a1424081dcfb3ad0ed056d. Latest consensus state: current_term: 1 leader_uuid: "ecab3c9182a1424081dcfb3ad0ed056d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecab3c9182a1424081dcfb3ad0ed056d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42637 } } peers { permanent_uuid: "9da53235e89c4810b4316ed69b319e5c" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 44047 } } peers { permanent_uuid: "c9b34a9d8c254e60a6f02e8d38cb6bb8" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 41289 } } }
I20260925 22:09:36.174678 27946 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9b34a9d8c254e60a6f02e8d38cb6bb8 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:09:36.175985 27970 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecab3c9182a1424081dcfb3ad0ed056d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ecab3c9182a1424081dcfb3ad0ed056d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecab3c9182a1424081dcfb3ad0ed056d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42637 } } peers { permanent_uuid: "9da53235e89c4810b4316ed69b319e5c" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 44047 } } peers { permanent_uuid: "c9b34a9d8c254e60a6f02e8d38cb6bb8" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 41289 } } }
I20260925 22:09:36.176087 27970 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecab3c9182a1424081dcfb3ad0ed056d [sys.catalog]: This master's current role is: LEADER
I20260925 22:09:36.176450 27946 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9b34a9d8c254e60a6f02e8d38cb6bb8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ecab3c9182a1424081dcfb3ad0ed056d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecab3c9182a1424081dcfb3ad0ed056d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42637 } } peers { permanent_uuid: "9da53235e89c4810b4316ed69b319e5c" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 44047 } } peers { permanent_uuid: "c9b34a9d8c254e60a6f02e8d38cb6bb8" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 41289 } } }
I20260925 22:09:36.176543 27946 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9b34a9d8c254e60a6f02e8d38cb6bb8 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:09:36.176851 27972 catalog_manager.cc:1347] Generated new cluster ID: c758e07408384a81883894a2c942a35a
I20260925 22:09:36.176916 27972 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:09:36.183038 27974 mvcc.cc:205] Tried to move back new op lower bound from 7333372625601601536 to 7333372625512230912. Current Snapshot: MvccSnapshot[applied={T|T < 7333372625601601536}]
I20260925 22:09:36.189442 27854 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.0.124" port: 41289 } attempt: 3
Sep 25 22:09:36 dist-test-slave-7mqr krb5kdc[27698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790374175, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.0.125@KRBTEST.COM for kudu/127.5.0.124@KRBTEST.COM
I20260925 22:09:36.198433 27856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:36.189678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.0.124:41289 (local address 127.0.0.1:41700)
0925 22:09:36.189725 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 22:09:36.189739 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 22:09:36.189793 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:09:36.189934 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:09:36.189935 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:09:36.190124 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:09:36.190172 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:36.190174 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.191103 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.191103 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:36.191926 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:09:36.191929 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.192445 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.192446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:09:36.192519 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:36.193086 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:09:36.193091 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:09:36.194619 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:09:36.197198 (+ 2579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:36.197199 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:36.197200 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:36.197465 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:36.197713 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:09:36.197714 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:09:36.197714 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:09:36.197787 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:09:36.198115 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:09:36.198115 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:09:36.198174 (+ 59us) client_negotiation.cc:769] Sending connection context
0925 22:09:36.198234 (+ 60us) client_negotiation.cc:240] Negotiation successful
0925 22:09:36.198331 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260925 22:09:36.198815 27934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:36.189772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41700 (local address 127.5.0.124:41289)
0925 22:09:36.189818 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:36.189821 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:36.189830 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:09:36.189852 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:36.189853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:36.189862 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:36.189911 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:36.190260 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.190998 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.192014 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:36.192314 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:36.192373 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:36.195225 (+ 2852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:09:36.195226 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:09:36.195227 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:09:36.195233 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:09:36.197089 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:36.197559 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:36.197561 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:36.197561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:36.197632 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:09:36.197865 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:09:36.197866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:09:36.197866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:09:36.198007 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:09:36.198074 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:36.198653 (+ 579us) server_negotiation.cc:299] Negotiation successful
0925 22:09:36.198729 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 22:09:36.200836 27854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9da53235e89c4810b4316ed69b319e5c: Bootstrap starting.
I20260925 22:09:36.201545 27854 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9da53235e89c4810b4316ed69b319e5c: Neither blocks nor log segments found. Creating new log.
I20260925 22:09:36.201825 27854 log.cc:824] T 00000000000000000000000000000000 P 9da53235e89c4810b4316ed69b319e5c: Log is configured to *not* fsync() on all Append() calls
I20260925 22:09:36.207322 27854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9da53235e89c4810b4316ed69b319e5c: No bootstrap required, opened a new log
I20260925 22:09:36.208397 27972 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:09:36.209116 27972 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:09:36.209368 27854 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9da53235e89c4810b4316ed69b319e5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecab3c9182a1424081dcfb3ad0ed056d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42637 } } peers { permanent_uuid: "9da53235e89c4810b4316ed69b319e5c" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 44047 } } peers { permanent_uuid: "c9b34a9d8c254e60a6f02e8d38cb6bb8" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 41289 } }
I20260925 22:09:36.209498 27854 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9da53235e89c4810b4316ed69b319e5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9da53235e89c4810b4316ed69b319e5c, State: Initialized, Role: FOLLOWER
I20260925 22:09:36.209576 27854 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9da53235e89c4810b4316ed69b319e5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecab3c9182a1424081dcfb3ad0ed056d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42637 } } peers { permanent_uuid: "9da53235e89c4810b4316ed69b319e5c" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 44047 } } peers { permanent_uuid: "c9b34a9d8c254e60a6f02e8d38cb6bb8" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 41289 } }
I20260925 22:09:36.209905 27854 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9da53235e89c4810b4316ed69b319e5c [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:09:36.211210 27828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ecab3c9182a1424081dcfb3ad0ed056d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9da53235e89c4810b4316ed69b319e5c"
I20260925 22:09:36.211309 27828 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9da53235e89c4810b4316ed69b319e5c [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:09:36.212152 27828 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9da53235e89c4810b4316ed69b319e5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecab3c9182a1424081dcfb3ad0ed056d in term 1.
I20260925 22:09:36.212220 27972 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ecab3c9182a1424081dcfb3ad0ed056d: Generated new TSK 0
I20260925 22:09:36.212390 27972 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:09:36.212524 27827 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9da53235e89c4810b4316ed69b319e5c [term 1 FOLLOWER]: Refusing update from remote peer ecab3c9182a1424081dcfb3ad0ed056d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:09:36.212783 27937 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ecab3c9182a1424081dcfb3ad0ed056d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9da53235e89c4810b4316ed69b319e5c" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 44047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:09:36.215040 27994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9da53235e89c4810b4316ed69b319e5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ecab3c9182a1424081dcfb3ad0ed056d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecab3c9182a1424081dcfb3ad0ed056d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42637 } } peers { permanent_uuid: "9da53235e89c4810b4316ed69b319e5c" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 44047 } } peers { permanent_uuid: "c9b34a9d8c254e60a6f02e8d38cb6bb8" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 41289 } } }
I20260925 22:09:36.215138 27994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9da53235e89c4810b4316ed69b319e5c [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:09:36.215184 27994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9da53235e89c4810b4316ed69b319e5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ecab3c9182a1424081dcfb3ad0ed056d. Latest consensus state: current_term: 1 leader_uuid: "ecab3c9182a1424081dcfb3ad0ed056d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecab3c9182a1424081dcfb3ad0ed056d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42637 } } peers { permanent_uuid: "9da53235e89c4810b4316ed69b319e5c" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 44047 } } peers { permanent_uuid: "c9b34a9d8c254e60a6f02e8d38cb6bb8" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 41289 } } }
I20260925 22:09:36.215224 27994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9da53235e89c4810b4316ed69b319e5c [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:09:36.215255 27994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9da53235e89c4810b4316ed69b319e5c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ecab3c9182a1424081dcfb3ad0ed056d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecab3c9182a1424081dcfb3ad0ed056d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42637 } } peers { permanent_uuid: "9da53235e89c4810b4316ed69b319e5c" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 44047 } } peers { permanent_uuid: "c9b34a9d8c254e60a6f02e8d38cb6bb8" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 41289 } } }
I20260925 22:09:36.215296 27994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9da53235e89c4810b4316ed69b319e5c [sys.catalog]: This master's current role is: FOLLOWER
W20260925 22:09:36.217448 27990 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9da53235e89c4810b4316ed69b319e5c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:09:36.217502 27990 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 22:09:36.243752 27934 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:09:36.242133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41704 (local address 127.5.0.124:41289)
0925 22:09:36.242178 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:36.242180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:36.243114 (+ 934us) server_negotiation.cc:407] Connection header received
0925 22:09:36.243221 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:36.243222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:36.243232 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:36.243263 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:36.243688 (+ 425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41704: BlockingRecv error: recv got EOF from 127.0.0.1:41704 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260925 22:09:36.244685 27782 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:09:36.242373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35998 (local address 127.5.0.126:42637)
0925 22:09:36.242410 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:36.242412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:36.244171 (+ 1759us) server_negotiation.cc:407] Connection header received
0925 22:09:36.244271 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:36.244272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:36.244282 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:36.244311 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:36.244622 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35998: BlockingRecv error: recv got EOF from 127.0.0.1:35998 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260925 22:09:36.245471 27858 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:09:36.242588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51018 (local address 127.5.0.125:44047)
0925 22:09:36.242624 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:36.242626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:36.244894 (+ 2268us) server_negotiation.cc:407] Connection header received
0925 22:09:36.245006 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:36.245007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:36.245019 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:36.245049 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:36.245399 (+ 350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51018: BlockingRecv error: recv got EOF from 127.0.0.1:51018 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260925 22:09:36.247745 27858 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:09:36.247028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51024 (local address 127.5.0.125:44047)
0925 22:09:36.247062 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:36.247064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:36.247255 (+ 191us) server_negotiation.cc:407] Connection header received
0925 22:09:36.247353 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:36.247354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:36.247370 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:36.247398 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:36.247701 (+ 303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51024: BlockingRecv error: recv got EOF from 127.0.0.1:51024 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260925 22:09:36.251798 27858 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:09:36.251044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51032 (local address 127.5.0.125:44047)
0925 22:09:36.251077 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:36.251079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:36.251302 (+ 223us) server_negotiation.cc:407] Connection header received
0925 22:09:36.251391 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:36.251392 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:36.251402 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:36.251429 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:36.251755 (+ 326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51032: BlockingRecv error: recv got EOF from 127.0.0.1:51032 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260925 22:09:36.255579 27858 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:09:36.254874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51046 (local address 127.5.0.125:44047)
0925 22:09:36.254915 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:36.254917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:36.255136 (+ 219us) server_negotiation.cc:407] Connection header received
0925 22:09:36.255221 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:36.255222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:36.255230 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:36.255257 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:36.255540 (+ 283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51046: BlockingRecv error: recv got EOF from 127.0.0.1:51046 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260925 22:09:36.261387 27858 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:09:36.260607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51056 (local address 127.5.0.125:44047)
0925 22:09:36.260643 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:36.260646 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:36.260896 (+ 250us) server_negotiation.cc:407] Connection header received
0925 22:09:36.260990 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:36.260991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:36.261000 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:36.261029 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:36.261344 (+ 315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51056: BlockingRecv error: recv got EOF from 127.0.0.1:51056 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260925 22:09:36.271449 27858 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:09:36.270543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51068 (local address 127.5.0.125:44047)
0925 22:09:36.270597 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:36.270600 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:36.270876 (+ 276us) server_negotiation.cc:407] Connection header received
0925 22:09:36.270979 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:36.270980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:36.270991 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:36.271021 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:36.271387 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51068: BlockingRecv error: recv got EOF from 127.0.0.1:51068 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260925 22:09:36.289669 27858 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:09:36.288717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51072 (local address 127.5.0.125:44047)
0925 22:09:36.288760 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:36.288762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:36.289062 (+ 300us) server_negotiation.cc:407] Connection header received
0925 22:09:36.289170 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:36.289171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:36.289183 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:36.289213 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:36.289610 (+ 397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51072: BlockingRecv error: recv got EOF from 127.0.0.1:51072 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260925 22:09:36.323877 27858 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:09:36.322977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51082 (local address 127.5.0.125:44047)
0925 22:09:36.323020 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:36.323022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:36.323291 (+ 269us) server_negotiation.cc:407] Connection header received
0925 22:09:36.323390 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:36.323391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:36.323402 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:36.323431 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:36.323814 (+ 383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51082: BlockingRecv error: recv got EOF from 127.0.0.1:51082 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260925 22:09:36.389923 27858 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:09:36.389242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51088 (local address 127.5.0.125:44047)
0925 22:09:36.389322 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:36.389324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:36.389371 (+ 47us) server_negotiation.cc:407] Connection header received
0925 22:09:36.389433 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:36.389434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:36.389445 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:36.389487 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:36.389836 (+ 349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51088: BlockingRecv error: recv got EOF from 127.0.0.1:51088 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260925 22:09:36.519753 27858 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:09:36.519065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51100 (local address 127.5.0.125:44047)
0925 22:09:36.519127 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:36.519129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:36.519211 (+ 82us) server_negotiation.cc:407] Connection header received
0925 22:09:36.519257 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:36.519258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:36.519268 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:36.519317 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:36.519685 (+ 368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51100: BlockingRecv error: recv got EOF from 127.0.0.1:51100 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 22:09:36.743435 27937 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecab3c9182a1424081dcfb3ad0ed056d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ecab3c9182a1424081dcfb3ad0ed056d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecab3c9182a1424081dcfb3ad0ed056d" member_type: VOTER last_known_addr { host: "127.5.0.126" port: 42637 } } peers { permanent_uuid: "9da53235e89c4810b4316ed69b319e5c" member_type: VOTER last_known_addr { host: "127.5.0.125" port: 44047 } } peers { permanent_uuid: "c9b34a9d8c254e60a6f02e8d38cb6bb8" member_type: VOTER last_known_addr { host: "127.5.0.124" port: 41289 } } }
I20260925 22:09:36.743600 27937 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecab3c9182a1424081dcfb3ad0ed056d [sys.catalog]: This master's current role is: LEADER
W20260925 22:09:36.777845 27858 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:09:36.777026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51102 (local address 127.5.0.125:44047)
0925 22:09:36.777100 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:36.777102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:36.777115 (+ 13us) server_negotiation.cc:407] Connection header received
0925 22:09:36.777140 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:36.777141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:36.777150 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:36.777214 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:09:36.777775 (+ 561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51102: BlockingRecv error: recv got EOF from 127.0.0.1:51102 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 22:09:37.148763 27959 catalog_manager.cc:1259] Loaded cluster ID: c758e07408384a81883894a2c942a35a
I20260925 22:09:37.148833 27959 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c9b34a9d8c254e60a6f02e8d38cb6bb8: loading cluster ID for follower catalog manager: success
I20260925 22:09:37.149901 27959 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c9b34a9d8c254e60a6f02e8d38cb6bb8: acquiring CA information for follower catalog manager: success
I20260925 22:09:37.150233 27959 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c9b34a9d8c254e60a6f02e8d38cb6bb8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 22:09:37.217950 27990 catalog_manager.cc:1259] Loaded cluster ID: c758e07408384a81883894a2c942a35a
I20260925 22:09:37.218032 27990 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9da53235e89c4810b4316ed69b319e5c: loading cluster ID for follower catalog manager: success
I20260925 22:09:37.218887 27990 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9da53235e89c4810b4316ed69b319e5c: acquiring CA information for follower catalog manager: success
I20260925 22:09:37.219193 27990 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9da53235e89c4810b4316ed69b319e5c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 22:09:37.293509 28065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:37.290989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51112 (local address 127.5.0.125:44047)
0925 22:09:37.291096 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:37.291098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:37.291150 (+ 52us) server_negotiation.cc:407] Connection header received
0925 22:09:37.291193 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:37.291194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:37.291243 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:37.291316 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:09:37.291482 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:37.291998 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:37.292754 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:37.292942 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:37.292991 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:37.293136 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:09:37.293260 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:09:37.293361 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:37.293377 (+ 16us) server_negotiation.cc:299] Negotiation successful
0925 22:09:37.293397 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":46,"threads_started":1}
I20260925 22:09:37.297668 28071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:37.295346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41706 (local address 127.5.0.124:41289)
0925 22:09:37.295472 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:37.295474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:37.295483 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:09:37.295529 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:37.295529 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:37.295561 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:37.295629 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:09:37.295728 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:37.296348 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:37.297014 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:37.297195 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:37.297247 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:37.297363 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:09:37.297453 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:09:37.297510 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:37.297546 (+ 36us) server_negotiation.cc:299] Negotiation successful
0925 22:09:37.297563 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":80,"threads_started":1}
I20260925 22:09:37.301916 28065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:37.299209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51122 (local address 127.5.0.125:44047)
0925 22:09:37.299358 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:37.299360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:37.299478 (+ 118us) server_negotiation.cc:407] Connection header received
0925 22:09:37.299588 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:37.299590 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:37.299602 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:37.299653 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:09:37.299765 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:37.300317 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:37.301127 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:37.301473 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:37.301546 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:37.301678 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:09:37.301705 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:09:37.301752 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:37.301822 (+ 70us) server_negotiation.cc:299] Negotiation successful
0925 22:09:37.301841 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260925 22:09:37.302219 28071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:37.299241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41710 (local address 127.5.0.124:41289)
0925 22:09:37.299281 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:37.299283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:37.299480 (+ 197us) server_negotiation.cc:407] Connection header received
0925 22:09:37.299700 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:37.299701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:37.299709 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:37.299779 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:09:37.299946 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:37.300414 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:37.301471 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:37.301780 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:37.301837 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:37.301979 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:09:37.302008 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:09:37.302055 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:37.302137 (+ 82us) server_negotiation.cc:299] Negotiation successful
0925 22:09:37.302159 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:09:37.303300 28078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:09:37.299261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36000 (local address 127.5.0.126:42637)
0925 22:09:37.299360 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0925 22:09:37.299362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:09:37.299376 (+ 14us) server_negotiation.cc:407] Connection header received
0925 22:09:37.299493 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:09:37.299495 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:09:37.299563 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:09:37.299648 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:09:37.300398 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:37.301170 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:37.302436 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:09:37.302725 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:09:37.302790 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:09:37.302979 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:09:37.303069 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:09:37.303143 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 22:09:37.303193 (+ 50us) server_negotiation.cc:299] Negotiation successful
0925 22:09:37.303217 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":55,"threads_started":1}
I20260925 22:09:37.309742 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 27711
I20260925 22:09:37.314814 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 27786
I20260925 22:09:37.319571 5121 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMOkht1/build/release/bin/kudu with pid 27861
2026-09-25T22:09:37Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2820 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5593 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (268339 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST